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FOREVJ  ORD 


The  Human  Factors  Technical  Area  of  the  Army  Research  Institute 
(AR1)  is  concerned  with  helping  users  and  operators  cope  with  the  ever 
increasing  complexity  of  the  battlefield  automated  systems  by  which  they 
acquire,  transmit,  process,  disseminate,  and  utilize  information.  In¬ 
creased  system  complexity  increases  demands  imposed  on  the  human  inter¬ 
acting  with  the  machine.  ARI's  efforts  in  this  area  focus  on  human  perfor¬ 
mance  problems  related  to  interactions  with  command  and  control  centers, 
and  on  issues  of  system  design  and  development.  Research  is  addressed  to 
such  areas  as  user-oriented  systems,  software  development,  information 
management,  staff  operations  and  procedures,  decision  support,  and  systems 
integration  and  utilization. 

An  area  of  special  concern  in  user-oriented  systems  is  the  improvement 
of  the  user-machine  interface.  Lacking  consistent  design  principles, 
current  practice  results  in  a  fragmented  and  unsystematic  approach  to 
system  design,  especially  where  the  user/operator-system  interaction  is 
concerned.  Despite  numerous  design  efforts  and  the  development  of  exten¬ 
sive  system  user  information  over  several  decades,  this  information  remains 
widely  scattered  and  relatively  undocumented  except  as  it  exists  within  and 
reflects  a  particular  system.  The  current  effort  is  dedicated  to  the 
development  of  a  comprehensive  set  of  Human  Factors  guidelines  and  eval¬ 
uation  criteria  for  the  design  of  user/operator  transactions  with  battle¬ 
field  automated  systems.  These  guidelines  and  criteria  are  intended  to 
assist  proponents  and  managers  of  battlefield  automated  systems  at  each 
phase  of  system  development  to  select  the  design  features  and  operating 
procedures  of  the  human-computer  interface  which  best  match  the  require¬ 
ments  and  capabilities  of  anticipated  users /opera tors. 

Research  in  the  area  of  user -oriented  systems  is  conducted  as  an 
in-house  effort  augmented  through  contracts  with  uniquely  qualified 
organizations.  The  present  effort  was  conducted  in  collaboration  with 
personnel  from  Synectics  Corporation  under  contract  MDA903-80-C-0094. 
The  effort  is  responsive  to  requirements  of  Army  Project  2Q263744A793, 
Human  Performance  Effectiveness  and  Simulation,  and  to  special  requirements 
of  the  U.S.  Army  Combined  Arms  Combat  Developments  Activity  (CACDA),  Fort 
Leavenworth,  Kansas. 


DESIGN  GUIDELINES  AND  CRITERIA  FOR  USER/OPERATOR  TRANSACTIONS  WITH  BATTLE¬ 
FIELD  AUTOMATED  SYSTEMS  VOLUME  V:  BACKGROUND  LITERATURE 


EXECUTIVE  SUMMARY 


Requirement : 

To  develop  a  comprehensive  set  of  human  factors  guidelines  and  criteria 
for  the  design  of  user/ope rator  transactions  in  battlefield  automated 
systems  for  use  by  human  factors  specialists  and  system  proponents, 
managers,  and  developers. 


Procedure : 

To  augment  data  collected  during  analyses  of  a  series  of  battlefield 
automated  systems,  selective  literature  relevant  to  the  development  of 
human  factors  guidelines  and  criteria  for  the  design  of  user/operator 
transactions  was  reviewed.  The  literature  review  effort  was  limited 
primarily  to  recent  ARI  publications  and  to  reports  containing  guidelines 
which  might  be  incorporated  into  the  final  product  of  the  project. 


Findings : 

This  literature  review  demonstrated  that  research  results  are  generally 
inadequate  to  support  design  of  good  user/operator  transactions  in  auto¬ 
mated  systems.  In  addition,  experimental  investigations  and  other  research 
efforts  have  not  kept  sufficient  pace  with  technological  development  to 
provide  an  adequate  data  base  for  generation  of  guidelines  and  criteria 
applicable  to  the  rapidly  changing  user/operator  transaction  environment. 


Utilization  of  Findings: 

Findings  from  the  analysis  of  individual  systems  may  be  useful  to 
proponents  in  specifying  user/operator  requirements  for  future  system 
evolution.  In  this  project,  the  findings  were  incorporated  in  a  data  base 
on  human  factors  requirements  which  provided  the  "real  world"  foundation 
for  development  of  the  provisional  guidelines  and  criteria  presented  in 
volume  IV  of  this  report.  The  provisional  guidelines  and  criteria  will  be 
utilized  as  the  basis  for  development  of  the  prototype  handbook. 
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INTRODUCTION 


Ire  '  Army  :  s  not  i  voly  enunqod  in  dew  hr  incut  o!  ,iutunutr.i  cyst  <-mr 

t  meet  ant  ic  ij  at  ed  requ  i  i  emei.t  s  ,  l  in- 1  u.i  i  i.q  bat  t  le t  i  e  1  d  missi-ir  ny:i  iv"ir:.' 
More  t  han  oe  automated  systems  r.n  rei.t  ly  in  a  n  nu-irt  dot  in  i  t.  ion,  di-vein-:  m. 
ct  :  roduot  .on  t!  aao  dot  end  upon  (  ho  .ii-si-n.  .md  accom;  1  i  shrm-nt  of  off<  '  i  v«  • 
tt-.ot .  oyo rat  or  t  lunsuct  ions; — t  he  usei/comput  or  interface,  t  i  on-dune:  ,  a:.-: 
t  i  n.n  ;  y.tos  — -t  o  :  ormi  t  of  root  ;  o-  ut  i  I  i  ;:at  ion  and  accom:  lishment  of  in  i  :  on 
:•  ie.-t  i  Vos  . 

!’he  Army  Kvs< arch  Ir.st  ituto  for  t  ho  Behavioral  and  social  Fc  i  oncer-  (A--: 
rovriis  so  i cut  i  f  i  <■  and  technical  su;  port  to  tiio  human  resources  needs  of  ? 
Army,  usina  t  echniques  attd  disc  ip  1  inos  of  behavioral  and  social  science 
(.V:  Jr.cr,  of  al ,  1  *>7rt,  p.  .  AKT  conducts  in-house  research  and  suiport; 

arant  and  font  ract  research  in  the  following  general  areas:  personnel  and 
manpower,  edueat  ion  and  training,  training  devices  and  simulation,  and  human 
factors  in  training  and  operat.  iona 1  systems.  This  research  is,  in  part,  in 
support  of  automated  systems  development. 

The  Human  factors  Technical  Area  of  ARI  is  concerned  with  effective 
ut  i ! i ration  of  battlefield  automated  systems  (BASa) ,  especially  because  of 
the  increasingly  complex  demands  of  systems  which  are  used  to  acquire,  trets 
mit,  process.,  disseminate',  and  utilise  information.  Tin1  Human  Factors  unit, 
is  particularly  concerned  with  the  demands  placed  or.  human  performance  relat 
to  tiie  human-comput  er  interaction.  API's  human  factors  research,  efforts  hav 
been  concerned  with  software  development,  t.opoqraphic  products  and  procedure 
tactical  symboloqy,  user-ori ented  systems,  information  manaqomont ,  staff 
operations  and  procedures,  decision  support,  and  sensor  systems  and  uti.lir.a- 
f  i  on . 

Because  the  ARI  human  factors  efforts  are  so  closely  akin  to  the  needs 
of  this  project,  some  of  the  pertinent  research  products  have  been  reviewed 
to  demonstrate  how  this  literature  can  contribute  to  the  development,  of 

'/feidner,  Joseph.  W< -s, -arcl;  t  homes  in  behnv i ovn I  ana'  hoc  i.ii  sciences.  I'.S. 
Army  Research  Institute  for  the  behavioral  and  social  sciences,  Alexandria 
Virginia,  l‘>7u. 
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uui>i<*l  inos  and  .  •valuation  criteria  for  the  design  of  u.sor/oi -orator  transact  io>.. 
;  1  .it  t  lofivlii  automated  systems.  Sevet.il  ci.uruct  .  y  i  st:  i  ■  -s  of  this  1  i  tern*  •• 

iwivw  should  be  made  clear: 

a.  Tiie  review  was  conducted  as  a  demonstrat  ion  of  how  Af.I  docu¬ 
ments  and  other  research  retorts  su; j ort  project  activities; 
no  at  tempt  has  been  made  to  integrate  or  contrast  results 
obtained  through  experimental  invent igai ions .  A  full  litera¬ 
ture  review,  to  be  conducted  during  the  second  phase  of  the 
project,  will  more  exhaust ; vely  incorporate  relevant  APT  and 
other  research  efforts. 

b.  With  few  exceptions,  only  very  recent  report s  have  seen 
included  in  the  review.  Much  earlier  work  also  has  signif¬ 
icance  for  the  development  and  presentation  of  human  factors 
guidelines  and  criteria. 

c.  Some  reports  have  been  included  because  they  contain  guide¬ 
lines  which  could  almost  directly  be  incorporated  into  the 
intended  final  product  of  the  study  effort;  others  are 
included  because  they  provide  information  which  car.  contri¬ 
bute  to  or  convert  to  guidelines  or  criteria. 

The  organizing  structure  for  presenting  the  ARI  literature  review  is 
that  employed  to  date  throughout  the  project  for  presenting  information 
about  the  systems  reviewed  and  for  the  development  of  guidelines  and  cri¬ 
teria — categories  of  design  features  affecting  user/operator  transactions. 

These  categories  are  listed  in  Table  1.  APT  research  reports  are  presented 
u:.  the  following  pages . 


1.  CONTROL  METHODS 
1 . 1  Command  Languages 

operating  Systems,  Inc.  MIQSTVRE:  An  experimental  online  language  for  army 
tactical  intelligence  information  processing .  ARI  Technical  Rep>ort  No. 
TR-78-A-2 5 ,  July  1978.  (AD  A064  323). 


In  its  search  for  ways  to  address  the  demands  for  increased  complexity 
in  the  user-machine  interface  of  BAS,  ARI  supported  this  effort  for  develop¬ 
ment  of  an  online  language  for  Army  tactical  intelligence  information  process¬ 
ing.  MIQSTURE  (Mixed  Initiative  Query  Structure  with  Task  and  User  Related 
moments)  is  an  attempt  to  provide  a  user-machine  dialogue  in  which  the  computer 
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6.  1  Standard  Terms 

6.2  Character  Sets  and  Labels 
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6.4  Abbreviation  and  Coding 
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has  an  act  ive  role  instead  ul  the  dialogue  being  in  it  iu*  «-d  oompb-t  ,-ly  i  y  • 
user.  Ml-.'STbKK  alb  >ws  the  computer  to  prompt  the  user  through  eomi  )•  x  *  a. 

•  tov  ivies  task  spec  i :  ic  effort  feedback ,  and  alerts,  the  as.ei  t «,  "<>t  "  s,: 

ivvrli'j  mu'it  :  roqressed  throuqh  -4  st  aqes  : 

a.  Ke  tu  i  cements  lief  ini  t  ion  based  on  doc  uitie  nt.at  io:.  and  interviews. 

b.  donee;  t  development  based  on  the  requirements  vie!'  it.it 

c.  I  reparation  of  detailed  design  spec  i  f  i  cat  ions  for  f  sub  *s 
v>f  tiie  lanquaue  to  demonstrate  operating  character  ist  i  ,.-s 
and  potential. 

d.  Evaluation  of  the  language  primarily  to  determine  which  desian 
goals  had  bee:,  reached  and  for  the  subset  for  pjery  i and 
manipulatin'!  tabular  data  in  the  data  base,  con;  ar  is-  >:  with 
tile  data  base  language  Sivdl'Eb. 

The  developers  claim  Mly'STURE  has  the  following  adva.n t  a -qeous  vfr.uruvt  .■ 
istics: 

a.  Adaptivity  to  change  through  tlie  use  of  sj  ocial  ly  designed 
mats  for  defining  new  capabilities  and  use  of  a  keyboard  sub¬ 
strate  language  to  provide  flexibility  for  new  and  emergent 
circumstances . 

b.  -Sensitivity  to  the  elements  and  stages  of  interactive  tasks 
through  separate  display  of  interaction  status  and  history, 
separate  incrementing  of  transaction  and  query  numbers,  a 
multistream  dialogue  which  allows  interleaving  of  tasks  at 
a  given  terminal,  and  a  capability  for  loading  "knowledge" 
of  detailed  task  structure  into  the  system  to  permit  the 
mixed  initiative  capability. 

Evaluation  of  MIQSTtJRE  was  carried  out  by  comparisons  anti  rat  ing.s  p  >  < > - 
vided  by  a  panel  of  experts  on  7  factors:  level  of  development,  training 
factors,  speed  factors,  power  factors,  accuracy  factors,  staffing  ease,  ami 
user  acceptability.  The  developers  conclude  that  the  preliminary/partial 
development  and  demonstration  of  MIQSTURE  has  achieved  the  desired  effeetiv 
ness  for  accomplishing  intelligence  processing  tasks. 


1 . 2  Menus 

No  i’fjottr-  it:  t  i  i  i  .  •  vMU'Muty  \\vn*  i  «•’/  i  i . 

1.7  Function  Keys 

iV  >  l  K.  • »  t '  t  t  :  ;  i  !  i  l  ;  .  1  :  >  cVl  t  «  'v  ]i  '  IV  \V«  •  l  t  1  hV  ll  '  •  1  . 

1 .4  Hybrid  Methods 

No  t opori s  in  this  cat eqory  won1  t  .•  v  .  ••.%••  >•  i . 

1  .5  Prompts/HELPS 

No  report s  in  {  5.  i a !  •  ’qoj  y  wi  t  •  -  v  i  • 

2.  DISPLAY  TORMAT 

2.1  Fi xed  Alphanumeri c  Displays 

Mo  ri'i  ortrt  ii:  this  oat eqory  wore  roviewe 

2.2  Variabl e-Length  Alphanumeric  Displays 

Baker,  J.P.,  .'■taco,  D.d.  ,  S  McKendry,  ■  t  •  it  :  on  ."  s': 

ixsc'ssmt'nt  of  t  .'it -  Inim.m  coin;  on<  nut .  AK1  Techt:  i  .n  !  Research  Not  c  No.  .  1.  , 
Auqust  19t>9  (AD  097  710). 

The  transform  or  oration  is  define.;  a  t  ran-  format  i  or.  of  raw  dat  i 

for  input  into  storage  devices.  This  study  «  xt  loro.1,  the  t  ine  and  oi  rors 
involved  in  mossniic  format  select  ion.  A  sum;  I"  of  14  militate'  and  civil  ia: 
personnel  working  on  the  'b 'S  (Tact  ical  ■  j  rl.it  i  nq  cyst  em)  development  select 
a  format  for  47  situation  messoqes  out  of  the  4  t  available  formats.  Pat  a 

col  looted  were:  t  imo  to  make  the  selection,  i  he  froruoncy  of  incorrect  tot 

mat  select  ion,  and  the  cert  i rude  ot  the  select  ion  (a  7  point  scale  from 
"absolutely  uncertain"  to  "absolutely  certain").  One  croup  used  a  iob-ai.i 
to  assist  in  the  format  .selection.  No  .1  i  f  f rt  cnees  wore  found  between  the 
job-aided  and  the  standard  q roups  on  any  ot 


flie  above  measur. 


Menu  t  ormat  si-lection  timo  was  49.8  seconds  j  er  message.  ■  't  :.«•»  data 
i  :>d  L  cat  ed  t  hat  during  teak  periods,  the  03  could  expect  to  receive  a  im-rsuM-- 
every  7c  to  seconds.  With  the  time  to  merely  select  a  message  foin.it  .o 

about  5c  si  -  cot  ids. ,  the  0  3  would  prickly  be  overloaded  it;  |  oak  j<Tiods. 

77  percent  error  rate  in  format  selection  was  obtained,  it:  the  select,  ion  of 
formats  for  reportedly  simple  messages.  A  detailed  analysis  of  the  orro-  • 
revealed  a  confusion  oi  outed  by  an  irrelevant  corrosj  ondence  between  ru  s: 
content  and  message  category  titles.  The  meat,  certitude  was  almost  idem 
for  the  two  groups  (5.5  for  the  job-aided  group  and  5.7  for  the  standard 

group),  indicating  that  the  particular  job  aid  provided  failed  to  aid  ;»-rhg- 

mance . 

The  authors  noted  (page  15)  that  "taken  together,  the  error  rate  an  i  t:.. 
time  data... lead  to  tiie  conclusion  that  the  transformation  process  i s  poten¬ 
tially  a  maior  problem  in  the  experimental  TOS . "  To  eliminate  format  select  in 
errors,  they  suggest  a  structured  training  program  or  a  set  of  mnemonic 
descriptions  for  message  titles,  drawing  attention  to  the  relevant  di  f fen.-n. 
in  the  p-urposo  of  the  various  message  formats.  They  speculate  that  the  most 
likely  error  (erroneous  selection  of  the  "relay"  message  format)  will  result 
in  a  communications  function  only  when  the  information  should  have  been  in;  ut 
into  the  data  base.  They  suggest  a  scrutiny  of  the  existing  formats  with  a 
goal  of  reducing  their  number.  (NOTE:  the  current  and  most  recent  successor 
system  to  TOS,  TCS/TCT,  in  using  only  four  basic  message  formats,  appears 
to  have  taken  their  advice  to  heart.) 

Nystrom,  C.O.,  and  Gividen ,  G.M.  Ease  of  learning  alternative  TOS  message 
reference  codes.  ARI  Technical  Paper  No.  326,  September  1978.  (AD  A061  697). 

In  work  closely  allied  to  that  of  Baker,  Mace,  and  McKendry  (1969)  and 
Strub  (1971),  these  researchers  set  out  to  assess  the  changes  to  the  DEVTOS 
formats  (in  the  formats  provided  for  TOS')  recommended  by  the  ad  hoc  committee 
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2  ,  . 

'  .  *  ,iii  1  i :  died  lor  that  pun  who.  Tins  e!  tort  was  dedicated  to  dctomunat  io:. 

i  i  :  r.io.o  change:;  cont  ributo  to  tin'  1  earn  ibi  1  i  ty  uf  1 1  it-  codes  developed  toi 

tih  T.if  formats.  The  article  is  also  valuable  to  this  project  lor  its 
•  t  esc:. t  at  ion  of  code  design  strateuics  and  instructional  :;trate<j  ics  assoc  la*  i 
Wit  ! I  Sj  ceil  1C  ,'Odc  di'MUIllS. 

The  rules  for  qenerat  i  ng  the  two  cod'  sets  at  «'*  as  follows: 

a.  DKVT.'S- 1  Iks  codes  — 11  aljhas,  1  tn.iri'  a  i  •  fits-  tw 

letters  ropre-tent  the  messaa.-  cat  ejnt  y  and  tic  i:  ;it  j  <•:  >  .  ■- 
sont.s  t.he  tyre  of  action.  The  first  lett.-i  ingai : 

friendly,  enemy,  and  "common"  mo. -sate  .-a  t  >  •■tor  i<  ■:  ,  wi  •  :.  c. 
exception,  I',  for  ai  rcraft  -relat  oil  mi  s.v  at  •  .»■  i  e:  .  The 

second  letter  is  arbitrarily  as .  i  ■  jj  .  e.  i  wit.-.::,  t  he  :  t  i  endly 

or  enemy  cateeor  ies,  but  i  s  always,  at.  "A"  for  t  he  .••ammo: i 
message.  The  action  code  (=1  uses  1  -  for  the  six  action 
codes,  but  exceptions  are  allowed  with  common  m»  usage. 1  where 
0,  8,  and  ret'laco  1,  2,  and  J. 

b.  TOS2  codes--*!  alphas  ( Lf.LL )  .  The  first  three  letters  form  an 
acronym  of  the  message  title.  The  fourth  letter  rej  resents 
the  action.  Examples  of  the  two  message  code  types  are  pre¬ 
sented  in  Table  2. 


Table  2 


Sample  DEVTOS  and  Tos “  Message  Codes 


1 

Code  Elements 

”1 

|  Message  Ti 

tie  Code 

Act  ion 

Code 

Message  Reference  R 
Code  1 

Message  Title 

DEVTOS 

TOS2 

DEVTOS 

TOS2 

DEVTOS 

TOS2 

G~2  Messages 

Enemy  Unit 

EA 

EUS 

1 

A 

EA1 

EUSA 

Status  -  Add 

Enemy  Situation 

EC 

ESI) 

4 

0 

EC  4 

ESDQ 

Data  -  Query 

G-3  Messages 

Air  Control 

EH 

ACM 

n 

C 

C 

X 

fj 

ACMC 

Mesaure  -  Change 
Spot  Report  - 

US 

USR 

5 

P 

US5 

USRT 

Special  Processing 

/In  anticipation  of  the  Development  Tactical  Operating  System  (DEVTOS)  by 
T<  >S 2  (the  Tactical  Operable  Segment  Tactical  Operating  System),  the  Combat 
Developments  Command  (CDO)  directed  the  formation  of  an  ad  hoc  committee  to 
(a)  recommend  revisions  to  the  formats  and  (b)  evaluate  these  revisions. 

The  commit fe<  also  established  the  following  recommendations:  (a)  con sol - 

idat  ion  of  message  formats,  (b)  use  of  variable  field  data  entry,  ( c >  map 
reference  coordinate  "packing,"  and  (d)  use  of  a  new  message  reference 
code  system. 
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Participants  we  re  i!(*  enlisted  and  -in  officer  ;  orsonnel  with  no  ;.r  ior 
experience  in  using  tho  fnunut...  Kn  1  i  st  ed  ;  er  .surne  1  wore  tan-iornly  assigned 
to  one  of  two  i'.J  mossaiie  code  nets  (],!.~  or  1. 1.1.1.)  .  officers  were  randomly 
assigned  to  one  of  two  -u  on.  o;  two  Gi  code  set:;  (also  I,!,-"  ot  1.1,1.],)  . 

There  were  J<>  mess.,,  i-  ;  >rn.i*  t  1 1  l.-s  ‘.7  '-3  mesratf  format  t.  it  les. 

Par  f  i  c  i  ran  r  s  ;  er  forme.:  at  a  .  "i:i;  nt  e-  t  m  i  .na  I  one  at  a  t  i  me .  Initially  ti.--y 
reviewed  a  tabulat  t  r. -s- -i, t r  ot  •; -sun -  •  i  ties  from  which  to  learn  mes¬ 

sage  format  titles.  in-  •  wa..  •  -  ■  ;  i  >v  Me  the  correct  message  reference 
code  from  the  message  t  it  1.  arid  a  t  ion  code.  Krror  feedback  informed  the 
: art icipant  of  the  correet  code  when  an  error  was  committed.  The  "learning" 
criterion  was  a  "perfect  |a'.s"--rniri-.:t  comj  let  ion  of  all  message  title.-,  it, 
the  message  set. 

The  following  were  t  he  ii.de;  endeid  variables: 

a.  Code  type  (l.I,~  or  i.i.I  i  )  . 

b.  G2  or  G3  list. 

c .  Rank  of  the  |  art  ici:  aid  ( ■  ■:  i  : . :  •  •  i  versus  off  ;  c.-r)  . 

Tho  dependent  var  i  •  li  be.  •*••••  .  a  follow.-;: 

a.  Percent  of  me  s  sag--  t  et  .-r.c.-  eo  les  input  with  one  oi  no  tv- 

errors  . 

b.  Number  of  "pons.-.."  t  hi  .  -a.::.  :  :«•  code  list  (learning  time). 

c.  Time  to  input,  a  lies  .an-  - -ode  (avei  aged  over  all  codes  and 
all  passes). 

d.  Analysis  of  error  rat  .  .  i  y  character  posit  ion  (for  officer 
participants  only)  . 

LLL1.  codes  were  cl-arly  ;u;--r  ior  to  codes  in  both  learning  rate  and 

error  rate.  The  mean  t.im-  to  in;  ut  i  message  reference  code  did  not  differ 
by  code  type:  a  time-  limit  of  2o  seconds  to  respond  was  jvrmitted  and  averagi 
response  time  was  about  5 .  r>  to  i,,r.  sccotuis,  Knl  isted  personnel  took  signifi¬ 
cantly  longer  to  learn  the  codes  than  officer  personnel  (lies;  ite  the  high 
enlisted  personnel  GT  score).  For  both  percentage  of  errors  and  average 
input  time,  differences  were  not  significant,  hut  onlv  marginally  so  at:  .053 
for  mean  percentage  of  errors. 


Analysis  of  errors  by  character  position  demonstrated  that  seven  of  the 
new  code's  characters  were  associated  with  excessively  high  error  rate:;.  Sour 
approaches  to  reducing  the  error  rate  and  instructional  strategies  associated 
with  these  approaches  are  suggested  by  the  authors  for  exploration: 

a.  Partial  code  revision--modif ication  of  only  those  codes 
associated  with  excessive  error  rate,  the  change  bringing 
these  codes  into  closer  acronymity. 

b.  Three  word  message  titles — forced  three  letter  acronyms. 

c.  Variable  length  acronymic  code — requires  message  titles  to 
consist  of  three  words  or  less  with  no  ambiguities . 

However,  involves  change  to  only  8  message  types  versus 
change  to  10  message  types  for  option  b. 

d.  Three  letter  code  based  on  first  and  last  words  of  the  mes- 
sag_e  tit  It  —  the  code  uses  the  first  two  letters  of  the  first 
word  and  t  he  Iirst  letter  of  the  last  word.  This  accommo¬ 
date.;  inesaagt  titles  of  two  words  or  more  and  requires 

-.a:  :•  *  •:  on]--  >  message  title. 

file  am  ;.<>.>  •  ■  •>  v.- d  a  !e  that  the  T OS'  message  reference  code  (L1.1.I.)  is 
clearly  bet  •  t  hut:  he  :  >|:VT*  »S  erode  (1.1,#)  and  they  suggest  a  further  experi¬ 
ment  and  ]  re.ii  a  t  hat  a|  |  roaches  b.  ,  c.  ,  and  d.  ,  would  all  ire  better  than 

a. )  . 

Most ■  s ,  K.L.,  and  Potash,  L.M.  Assessment  of  Abbreviation  methods  for  automate* 
fa  ■'  :  .•.•/stems.  ARI  Technical  Report  No.  398,  August  1979.  (AD  A077  840). 


A  ;  i.ui  .  :  a  continuing  effort  to  provide  efficient  vocabularies  and 
mes -.age  ,.t  t  ik'  are.,  for  BA:  s,  AKI  evaluated  alternative  methods  for  creating 
ail  reviat  ion-  f  military  terms.  Moses  and  Potash  establish  tire  following 
crite;  ia  f,,,-  aUifevi.it  ions:  (|  .  1) 

a.  yuirkly  and  easily  discriminated  from  other  abbreviations  in 
a  j  art i cul ar  syst em. 

i).  Kasily  decoded  and  remeinbered. 

c.  Compatible  witii  a  variety  of  different  system  configurations. 

Their  investigation  focused  on  the  design  and  evaluation  of  procedures 
for  creating  abbreviations;  their  review  of  the  literature  led  them  to  conclude 
that.:  "Clearly,  a  variety  of  suggestions  exist  for  shortening  single  as  well 
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us  multiple  word  items.  However,  no  genera 1 izabl o  recommendation  lias  emerged 
to t  methods  which  generate  abbreviations  that  are  preferred,  easily  decoded. 


and.  used  with  minimal  learning."  (p.  2) 

Participants,  60  enlisted  military  personnel  with  diverse  backgrounds, 
performed  three  tasks: 

a.  Preference  rati ng-- j udging  how  well  single  military  terms  are 
represented  by  abbreviations  generated  under  the  following 
five  methods: 

1.  tlata  Element  Dictionary  (DED) --representing  current  Army 
practice. 5 

2.  Simple  truncation — starting  from  the  right  end  of  the  word, 
dropping  off  characters  until  the  required  length  is 
obtained. 

3.  Truncation — 2nd  letter  out — elimination  cf  the  2nd  char¬ 
acter  followed  by  simple  truncation,  as  above. 

4.  Contraction — vowels  out — retention  of  the  1st  and  2nd 
characters,  removal  of  vowels  and  H,  W,  Y  (from  right 
to  left)  until  required  length  is  obtained,  supplement¬ 
ing  with  simple  truncation,  as  necessary. 

5.  Contraction — frequent  letters  out — retention  of  the  1st 
character,  elimination  of  letters  (from  right  to  left)  on 
the  basis  of  frequency  of  occurrence  (highest  frequency 
eliminated  first)  until  the  required  length  is  obtained. 

b.  Decoding — writing  the  original  term  from  the  abbreviation.  60 
abbreviations  were  generated  on  the  basis  of  the  5  methods 
described  above.  (Different  terms,  random  assignment  of  terms 
to  blocks  of  participants,  etc.,  permitted  a  modified  Latin 
Square  design.) 

c.  Encoding — generation  of  an  abbreviation  for  each  of  60  terms; 

30  terms  were  new,  30  were  used  in  task  a.). 

For  task  a. ,  preference  rating  on  a  10-point  scale,  showed  that  abbre¬ 
viations  prepared  by  contraction — vowels  out  and  by  simple  truncation  were 


V'l’he  DED  is  the  result  of  the  TRADOC  Data  Element  Standardization  Program 
and  was  developed  to  meet  the  needs  of  the  Tactical  Operable  Segment 
Tactical  Operations  System  (TOS2),  the  All  Source  Analysis  System  (ASAS) , 
and  the  Tactical  Fire  Control  System  (TACFIRE) . 
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preferred  over  other  methods,  with  the  PEP  and  truncation— acc-ond  letter  out. 
being  least  preferred.  For  task  b. ,  decoding,  liberal  scoring  (allowing 
spelling  errors)  and  strict  scoring  were  applied  separately  to  previously 
seen  items  (in  task  a.)  and  not  previously  seen  items.  Five  factor  ANoVA 
(based  on  the  modified  Latin  Square  design)  for  the  liberal  and  strict 
scores  and  comparison  of  mean  scores  supported  simple  truncation  as  the 
superior  method  with  the  DED  being  the  worst  technique  overall.  For  encod¬ 
ing  task  c.),  simple  truncation  again  appeared  a  frequently  used  technique; 
but,  the  DED,  in  contrast  to  tasks  a.  and  b. ,  scored  well. 

These  results  support  simple  truncation  as  the  method  for  providing  a 
high  percentage  of  acceptable  abbreviations  for  use  as  data  element  codes  in 
BASs.  The  authors,  however,  in  exploring  user/use  variations  of  these  results 
and  in  playing  these  results  against  those  obtained  by  other  researchers, 
offer  the  following  tempered  conclusions: 

Simple  truncation  is  an  easy-to-use  method  that  generates  a  high 
proportion  of  consistently  preferred  and  easily  decodable  abbre¬ 
viations  for  single  words.  This  method  should  benefit  system 
designers  by  reducing  the  time  needed  to  produce  many  good  abbre¬ 
viations  for  use  as  data  element  codes.  Such  abbreviations  pre¬ 
sumably  should  benefit  users  by  reducing  input  time  and  errors  in 
interactions  with  battlefield  systems.  However,  the  abbreviations 
produced  by  simple  truncation  are  not  intended  to  replace  commonlu 
accepted  abbreviations  and  arc  not  likely  to  be  judged  acceptable 
in  all  cases.  (p.  14) 

2 . 3  Graphic  Displays 

No  reports  in  this  category  were  reviewed. 

2.4  High! i ghti ng 

No  reports  in  this  category  were  reviewed. 

3.  DATA  ENTRY  AND  HANDLING 

3 . 1  Information  on  Legal  Entries 

No  reports  in  this  category  wore  reviewed. 
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3.2 


Un b urdening  of  Input 


Alderman,  I.N.,  Ehrenreich,  S.L.,  and  Bindewald,  R.  Recent  AH1  kesea / c/j 
Battlefield  Automated  Systems.  (In  press.) 


(The  authors'  abstract  follows.) 


This  paper  reviews  ARI  research  designed  to  improve  the  data  entry  pro¬ 
cess.  The  first  and  second  section  of' the  paper  describes  the  data  entry 
process  in  general  as  well  as  in  the  context  of  a  specific  battlefield  auto¬ 
mated  system,  the  Tactical  Operating  System  (TOS) .  Because  it  was  used  as  a:, 
exemplar  of  the  data  entry  process,  TOS  played  an  important  role  in  the 
development  of  improved  data  entry  procedures.  The  third  section  of  the 
paper  reviews  the  findings  and  conclusions  of  the  many  ARI  research  projects 
concerned  with  data  entry.  Among  the  areas  covered  in  ART's  research  pro¬ 
gram  are: 

a.  How  to  format  and  display  data  entry  information. 

b.  What  safeguards  can  be  developed  to  reduce  the  number  of 
operator  errors  made  and/or  accepted  by  the  system. 

c.  What  kinds  of  operator  job  aids  can  be  developed  to  improve 
performance . 

d.  How  to  improve  operator  training. 

e.  How  to  make  the  system's  message  codes  easier  to  use  and  more 
memorable . 

f.  How  to  improve  the  design  of  keyboards. 

The  forth  section  of  the  paper  reports  on  efforts  to  analyze  the  cause  of 
operator  errors.  This  section  also  discusses  the  development  of  a  simulation 
of  the  data  entry  process.  The  simulation  is  intended  to  facilitate  system 
design  by  permitting  the  inexpensive  evaluation  of  alternate  data  entry 
procedures.  The  fifth  section  presents  a  general  discussion  of  the  problems 
that  have  been  encountered  by  the  ARI  research  program.  Also  included  here 
is  a  discussion  on  how  this  program  might  be  improved  in  the  future.  The 
final  section  of  the  paper  summarizes  the  operational  implications  of  ARI ' s 
research  results. 
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Grunda,  T.M.  An  application  of  human  Factum i  concepts  to  an  interact  ive 
computer i  .'i'.J  [n'r&ormol  n  mi  rd- kooi  ■  i  na  system.  AR !  Research  Report  Mo. 

12.53,  January  198u. 

(The  author's  summary  follows.) 

To  reduce  the  burden  of  inefficient  record-keeping  systems,  the  U.S. 

Army  plans  to  convert  many  of  its  manual  and  s>  -mi  automatic  record-keeping 
systems  to  interactive,  real-time  systems.  To  obtain  the  optimal  level  of 
hardware/software  complexity,  to  meet  operator  and  user  requirements,  ar.d  to 
keep  system  costs,  system  comi  lexity,  and  operator  training  and  skill  require¬ 
ments  at  a  reasonable  level,  a  careful  analysis  and  application  of  appro¬ 
priate  behavioral,  man-computer  interface  guidelines  are  required.  A  working 
version  of  a  potential  interactive  real-time  record-keeping  system  (the 
Standard  Jnstallation/Division  Personnel  System,  SIDPERS)  was  created  to 
provide  a  hands-on  demonstration  of  input/output  procedures  and  software 
techniques  that  can  provide  assistance  to  o:  or ators/users . 

The  behavioral  techniques  and  procedures  in  the  demonstration  (e.g., 
feedback,  error  detection  and  correction,  prompting,  variable  entry  format, 
variable  input  modes)  were  integrated  with  user  and  operator  requirements  to 
produce  an  efficient  interactive  record-keeping  system.  It  was  successfully 
demonstrated  that  the  human- factored,  interactive  system  aided  several  types 
of  users  in  a  variety  of  ways.  The  system  functioned  as  an  instructive  aid 
to  the  inexperienced  user  and  as  a  memory  aid  to  the  experienced  user  by 
informing  the  user:  where  to  find  data  and  information;  how  to  input  the 
data;  when  errors  occurred,  what  type  of  errors  they  were  and  how  to  correct 
them;  what  certain  terms  meant;  and  what  inputs  were  acceptable  to  the  computer. 
The  computer  served  as  a  retrieval  clerk  for  those  users  who  needed  access  to 
information  residing  in  the  computer.  The  computer  also  acted  as  an  organizer 
for  the  processing  of  SIDPERS  transactions,  retrieval  of  information,  and 
alteration  of  the  transaction  cletk  displays. 

Fields,  A.F.,  Maisano,  R.E.,  and  Marshall,  C.F.  A  •.ompnr.it ive  analysis  of 
methods  for  tactical  data  inputting .  ARI  Technical  Paper  No.  327,  September 
1978.  (AD  A060  562). 

This  study  extends  ARI's  efforts  to  improve  data  input  beyond  its  pre¬ 
vious  explorations  regarding  the  man-machine  interface  (Alderman,  1976), 


individual  i  xed  t  raining  techniques  (bade,  Fields,  and  Alderman,  1  <7n)  , 
computer  i  rompt  inq  and  instruct  ion  (St  rub,  1  »7r0  ,  and  using  on- 1  ine  in;  utt 
with  verit'ieat  ion  (Strut-,  l '.‘71 )  .  In  contrast  to  tin  oarl  ior  efforts,  how 
tiiis  st  tidy  focuses  on  inputt  inq  procedures  as  these  affect  oj  or.it  or  s; 
accuracy . 


Four  methods  of  data  input  into  a  Tos  format,  each  us inq  automat  i 
tabhinq,  were  studied: 


a.  Typing  codes  into  the  message  format. .  Message  format  a:  i  care  or. 
the  screen;  operator  fills  in  appropriate  codes  referring  to  a 
dictionary  of  valid  codes,  as  needed.  e'omj  utor  rejects  illeaal 
entry;  operator  must  provide  valid  entry  to  continue.  Cursor 
moves  to  next  item  when  correct  entry  made. 

b.  Typing  witii  error  corrector .  Same  as  typing,  above,  excel  t 
computer  forms  a  series  of  hypotheses  about  the  error  and  the:., 
having  "decided"  on  the  correct  entry,  the  com)  utor  presents 
the  entry  on  the  screen.  If  the  operator  accepts  the  entry, 

it  is  input,  by  a  key  to  signal  acceptance.  If  not  accepted, 
the  operator  has  a  key  to  signal  retyping  of  the  entry.  If 
the  machine  cannot  form  a  hypothesis,  an  error  message  aj  pears 
on  the  screen  and  the  operator  retypes  an  entry. 

c.  Menus ■  Map  coordinates ,  dates,  and  cardinal  numbers  are  tyi  ed 
in.  All  other  data  entered  by  selecting  the  item  from  an 
alphabetically  or  logically  ordered  menu  via  a  trackball. 

When  typing  required,  instruction  to  type  replaces  the  menu. 

If  invalid  entry  typed  in,  item  rejected  and  valid  entry  pro¬ 
vided  as  in  data  input  type  a.,  above. 

d.  Typing  with  autocomplot.ion  and  English  option.  Same  as  ini  ut 
type  a.  with  the  following  exceptions: 

I.  The  English  definition  can  be  entered  instead  of  the 
code . 

II.  When  operator  feels  enough  characters  are  entered  to 
identify  the  entry,  depression  of  a  key  causes  the 
autocomp  1  et. i  on  program  to  take  over.  If  unique  match 
made  within  the  valid  entry  list,  the  autocompletion 
feature  finishes  the  entry.  If  no  unique  match  pos¬ 
sible,  the  proqram  asks  for  more  characters. 

The  measures  (dependent  variables)  were  format  completion  time,  the 
number  of  errors  per  format,  the  frequency  of  use  of  the  autocompletion  func¬ 
tion,  the  frequency  of  use  of  full  English  instead  of  codes,  the  frequency  of 
backspacing  and  typing  over,  and  the  stated  preference  among  the  input  method 
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Part  icij  ant  s  wore  I!  enlisted  personnel  randomly  assigned  to  t  in-  four 
ini  at  groups.  Thirty-six  free  text  messages  were  divided  into  four  sots  of 
nine  messages  each,  with  each  message  set  balanced  for  typos  of  subject, 
sources,  restrict  ions,  unit  identification,  and  difficulty  during  pilot  test¬ 
ing.  Each  first  message  of  each  message  set  was  a  practice  message. 

i'n  the  basis  of  both  mean  number  of  errors  per  message  and  fewest  errors 
per  input  method,  the  input  types  ranked  (from  best  to  worst)  as  follows: 
menus,  typing  with  error  corrector,  straight  typing,  and  typing  with  auto¬ 
completion.  Differences  among  the  input  methods  were  significant.  No  dif¬ 
ferences  were  found  among  input  methods  on  the  basis  of  mean  time  to  input  mes¬ 
sages.  On  the  basis  of  fastest  time,  the  input  types  ranked  as  follows-:  typiny 
with  error  corrector,  straight  typing,  and  menu  selection.  (The  authors'  speed 
prediction  was  as  follows:  autocompletion,  menu  selection,  error  corrector,  and 
straight  typing.)  When  preferences  were  compared  to  performance  time  and  errors 
(using  a  coefficient  of  agreement),  there  is  chance  agreement  or  a  slight  nega¬ 
tive  agreement.  Except  for  input  by  menu  selection,  backspacing  was  used  with 
approximately  the  same  frequency  across  participants  for  correcting  an  entry, 
correcting  spelling,  or  correcting  an  invalid  entry.  In  about  one-third  of  its 
attempts,  the  error  corrector  could  not  arrive  at  an  entry.  Participants  used 
the  autocompletion  feature  only  for  better  than  only  one-half  of  their  possible 
opportunities;  the  autocompletion  of  codes  was  incorrect  about  4  percent  of 
the  time  and  autocompletion  of  English  was  incorrect  a  little  better  than  9 
percent  of  the  time. 

The  authors  conclude  that  menus  should  be  used  for  data  input  since  they 
permit  fewer  errors  and  do  not  significantly  increase  input  time.  Provision 
of  a  menu-based  system  with  a  menu-override  option  for  experienced  users  is 
suggested.  Error  correctors,  autocompletion  and  full  English  option  are  not 
warranted  for  general  use,  in  their  judgment. 

3.3  Interrupts  and  Work  Recovery 

No  reports  in  this  category  were  reviewed. 

3.4  Manipulating  Stored  Data 

No  reports  in  this  category  were  reviewed. 
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4.  MESSAGE  COMPOSITION  AIDS 


4.1  System  De  s  i  c|n  Features 


•m.'V  iow  Marc n  1'*  /^. 


(The  author's  brief  of  the  report  follows.) 


hequ  i  roment. : 

This  research  was  conductor,  as  a  human  factors  ovaiuut  ion  •:  *  :.»•  2'art  i 

Fire  Direction  system  (TACFIKK)  command  and  control  system  for  * he  field 
artillery.  This  resort  su;  p  lements  the  Tv 'AT  A  f  '!>t-  test  rsr  .  :  t  ;  i«- 
vides  a  human  factors  evaluat  ion  of  equipment,  tasks  and  operat  inu  ;  r<  wodur 
training,  and  personnel  selection  requirements. 

Procedure : 

A  variety  of  techniques  were  used  in  this  human  factors  evaluation, 
'questionnaires  were  developed  and  administered;  these  addressed  sjecific 
human  factors  issues.  Those  quest ionnaires  wore  suj  i  lamented  by  interviews 
and  by  pertinent  data  from  TCATA  quest ionnai res  and.  data  collection  forms. 
Performance  assessments  were  also  obtained  for  individual  operate”  s  at.  the 
Artillery  Control  Console  and  on  the  Digital  Message  Device.  Personnel 
records  and  formal  course  grades  were  used  to  ai.ulyue  personnel  selection 
requirements. 

Findings : 

The  battalion  S-280  shelter  is  regarded  as  unacceptable  by  battalion 
Fire  Direction  Center  personnel.  The  major  problem  areas  are  the  shortage 
of  space  within  the  shelter,  the  configuration  of  equipment  within  this 
limited  spiace,  the  quality  of  the  ail  ,  and  the  noise  level.  Noise  levels 
are  in  excess  of  MIL-KTD-147-1A. 

With  the  exception  of  the  Digital  Message  Device,  and  the  Digital  Plot*. 
Map,  there  is  widespread  acceptance  of  individual  TACFIKK  equipments. 

Although  operators  maintain  that  their  tasks,  on  the  average,  are  easy 
the  consensus  of  operators  is  t hat  TACFIKK  training  must  be  conducted 


’r»  r 

Ut'.’.t  1 

!  y  i  r  s 

k  i 

ill:- 

aft-  to 

he 

mu  i  n  t  u  i  tied  . 

k 

:.-it  i 

mute.-:  o 

f  time 

rooyj  i 

red  -O 

ra  i 

avt 

'rag.-d 

a: 

»ulit 

a 

wo  ok  at  X.  ho 

com;  at. 

or  kir»* 

I )  i root 

i  or. 

'enter  u:. 

'a  r  i 

aide 

Konn.it 

r 

ha:; 

.age  Ksitr 

Device  site;; 

ai. 

d  1 

■  in  y  a 

w*a..*k  at 

i  < ;  i 

tal  - 

aco 

ic'r  Sit 

M( .  »*  •(  *v*  r 

t 

ind i cat  ions 

aro 

th 

at  more 

■  cm  ok  a.- 

■sis  n  o 

■ode  to  r 

•lav 

t\i  or 

;  muir.t. 

,a:;c 

o*  t  rairii 

ng 

• 

”,  OI 

'at  ort* 

I 

lO  u 

y,c  trio  s 

tandard  (OWERT 

V) 

key 

hoar  l 

hould  > 

0 

il 

ow  to  ty 

The  Army  classification  Battery  tjpears  to  provide  a  cost-effective  meats  of 
selecting  i r . ■  1  i vi duals  for  TACK Ike  schooling. 

LI  ir.ut  ior.  of  Finding:; : 

The  f  i  tidings  of  tit  i  s  ret  ort  will  serve  as  the  human  factors  input  to 
TCATA  an.d  'TEA  for  their  evaluation  of  the  TACK  I  RE  system.  These  findings 
will  also  he  sent  to  the  Army  Field  Artillery  School  (USA;- 'AS)  for  their 
impact  on  training  and  personnel  selection  requirements. 


4.2  Fo rma t_f o r  A1  ph an ujjie^rj c  Messages 

No  reports  in  this  category  were  reviewed. 

4 . 3  Graphic  M es sage s 

3ersh,  P. ,  Moses.  F.L.,  and  Maisano,  R.E.  Investigation  of  the  strength 
of  association  between  graphic  symbology  and  military  information .  ARI  Tec;.- 
r.ical  Pater  No.  324,  September  1978.  (AD  A064  260). 

The  authors  contend  that  dissatisfaction  with  conventional  symbology 
drives  invest igat ion  of  symbology  in  terms  of  "clarity,  simplicity,  consistency 
and  adequacy  for  computer  generation"  (page  1)  and  that  "there  is  little  or  no 
empirical  evidence  available  for  accepting  any  new  approaches  or  for  retaining 
conventional  symbology"  {ibid).  Their  investigation  is  an  attempt  to  allow 
common  cultural  influences  to  form  more  or  less  stereotypical  associations-- 
"Natural"  associations,  in  their  terminology.  They  explored  the  association 
of  simple  graphic  codes  or  symbols  with  verbal  concepts. 

The  graphic  symbol  set  was  kept  simple  and  consisted  of: 

a.  Seven  symbol  sets  (e.g.,  circles,  lines,  colors,  shading, 
bars)  with  each  set  varying  in  a  single  characteristic 
(e.g.,  size  of  circle,  number  of  lines,  width  of  bars). 
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b. 


Fight  geomet. r  1  c  forms  (diamond,  trapezoid,  rectangle,  circle 
triangle,  paral lelogram,  and  ellipse). 


c.  bight  "stick"  symbols  (e.g.,  arc,  bracket,  arrow,  cross). 

The  verbal  concepts  explored  included: 

a.  bight  information  categories  ( importance,  accuracy,  firepower, 
unit  level,  friend/enemy,  range,  and  concentration). 

b.  Nino  military  branch  designations  (infantry,  armor,  field 
artillery,  mechanized  infantry,  signal,  engineer,  air  defense 
artillery,  cavalry,  and  aviation). 

c.  Three  general  military  function  terms  (service  support  unit, 
maneuver  unit,  fire  support  unit). 

Participants  were  two  groups  of  enlisted  personnel  (114  and  137  persons 
respectively)  with  limited  prior  exposure  to  military  symbology.  The  first 
group  was  presented  a  set  of  graphic  symbols  followed  by  a  set  of  verbal  con¬ 
cepts.  Their  task  was  to  rank  order  the  verbal  concepts  according  to  how  well 
they  represented  the  symbol.  The  second  group's  task  was  the  reverse  of  the 
first  group's;  i.e.,  to  rank  order  the  symbols  according  to  how  well  they 
represented  the  concepts. 

Four  criteria  were  established  to  determine  the  strongest  symbol -concept 
associations: 

a.  A  statistically  best  mean  rank  (i.e.,  statistically  different 
from  any  other  association) . 

b.  A  mean  rank  statistically  better  than  any  other  association. 

c.  A  first  place  ranking  by  the  greatest  number  of  participants. 

d.  Fulfillment  of  either  a.  or  b.  and  of  c.  for  both  symbol-con¬ 
cept  and  concept-symbol  associations. 

Only  three  symbol-concept  associations  were  found  to  be  high  strength 
and  no  ambiguity: 

a.  Numerosity  (the  number  of  lines)  and  unit  length — which  sup¬ 
ports  the  current  military  coding  scheme  for  designating 
units. 

b.  Color  and  danger — other  strong  color  associations ;  e.g.,  with 
friend/enemy,  accuracy,  importance  indicate  that  color  symbol¬ 
izes  a  broader  concept  than  danger,  perhaps  "threat". 
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c.  Square  and  service  support — this  association  is  much  stronger 
in  tlie  concept -symbol  direction  than  in  the  symbol-concc] t 
d i rect ion . 

Moderate  and  minimal  levels  of  association  were  also  examined  but  th'-y 
provide  no  clear-cut  guidance  for  our  study  with  respect  to  guidel  it and 
criteria  for  user/operator  transactions  with  BASs  in  that: 

a.  Results  of  the  exploration  are  not  without  ambiguity/confus ion . 

b.  The  study  approach  limits  findings  to  specific  condition:; 
explored  rather  than  to  general  demonstration  of  design 
principles . 

The  authors  suggest  that  their  ranking  method  is  appropriate?  for  prelim¬ 
inary  comparisons  and  that  their  data  provide  suggestions  for  use  in  modify  mu 
current  symbology  systems.  They  also  suggest,  however,  that  Thurston's  juirmi- 
comparison  procedure  would  provide  more  refined  measurement  than  theirs-- 
albeit  at  increased  time  and  cost. 


5.  DATA  RETRIEVAL  ASSISTANCE 

5 . 1  Query  Method 

No  reports  in  this  category  were  reviewed. 

5 . 2  Query  Structure 

Ehrenreich,  S.L.  Query  language;  design  rt'commendat  ions  derived  iron;  the 
human  factors  literature,  ARI  Technical  Report  No.  484  (in  press). 

(The  author's  abstract  follows.) 

The  existing  human  factors  literature  on  query  languages  is  both  sparse  and 
scattered.  This  paper  seeks  to  collect  and  review  that  literature.  The  first 
section  of  the  paper  introduces  the  subject  of  query  languages.  In  the  second 
and  third  sections,  the  topics  of  natural  and  formal  query  languages  are  respec¬ 
tively  discussed.  These  two  types  of  query  languages  are  reviewed  with  the  objec. 
tive  of  determining  their  potential  for  expanding  the  population  of  computer  usei 
The  fourth  section  considers  some  general  issues  pertinent  to  both  types  of 
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logical  quantifiers,  the  user's  concept  of  data  organization,  mixed  ini  i  i  at  i 
dialogues,  and  the  use  of  abbreviations.  Methods  for  experimentally  evalua' ire 
specific  query  language  fi>atures  and  researcti  on  person-to-person  commas  i  cat  i  o: 
are  also  discussed  hero.  To  focus  the  findings  reported  in  the  i  receding  sec¬ 
tions,  the  fifth  section  summarizes  the  implications  of  the  research  performed 
to  date.  Next,  the  sixth  section  presents  possible  new  research  which  would 
be  of  value  to  the  designers  of  Army  tactical  information  systems.  The  j  a; or 
concludes  with  two  appendices.  Appendix  A  discusses  human  factors  review 
papers  concerned  with  tire  design  of  interactive  systems.  Appendix  B  presents 
a  compendium  of  design  recommendations  directed  towards  the  system  designer . 

(Appendix  B  is  replicated  below.) 


APPENDIX  B 

QUERY  LANGUAGE:  A  COMPENDIUM  OF  DESIGN  RECOMMENDATIONS 

These  recommendations  were  compiled  from  the  literature  review  that  is 
presented  in  the  main  body  of  this  paper  and  from  additional  sources.  In 
some  instances,  the  recommendations  that  are  presented  here  go  beyond  what 
can  be  empirically  substantiated.  These  recommendations  are  not  to  be  con¬ 
sidered  immutable.  Instead,  they  represent  the  author's  opinion  as  to  what 
guidelines  might  be  thoughtfully  offered  at  the  present  time  to  a  system 
designer . 


Recommendations:  General 


Data  Organization 


1.  The  organization  of  the  data  base  that  is  presented  to  the 
users  should  match  the  organization  perceived  to  be  natural 
by  the  users.  The  users'  natural  organization  can  be  dis¬ 
covered  through  experimentation  or  by  survey. 

2.  Casual  users  should  not  be  presented  with  a  multitude  of  models 
for  representing  the  data  base.  A  single  representation  of 
the  data  base  should  be  sufficient  for  the  total  range  of 

user  needs.  A  multiplicity  of  data  base  structures  only  tends 
to  confuse  the  casual  user. 
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^uant  i  f  iers 

3.  A  quei  y  language  should  minimize  the  use  of  quantificat  ion 
terms  (o.q.,  "some,"  "all").  People  have  qreat  difficulty 
in  usinq  quantifiers  unaml ..  i quously .  Exceptions  to  this 
rule  are  the  quantifiers  "no"  and  "none."  When  quantifiers 
are  required,  the  system  should  have  the  user  choose  the 
desired  quant i f i eat i on  statement  from  a  set  of  statements 
that  are  designed  to  maximize  their  distinctiveness. 

Evaluat ing  Language  Options 

4.  Test  major  query  language  features  prior  to  adopting  them. 
The  text  of  this  paper  provides  a  description  of  experi¬ 
mental  procedures  that  can  be  used  in  deciding  between 
alternative  design  options. 

Feedback  of  the  ^juerv 


5.  Trior  to  the  execution  of  a  user's  query,  the  computer 

should  rephrase  the  query  and  display  it  for  user  accep¬ 
tance.  This  assures  that  the  user's  intended  meaning 
lias  been  correctly  interpreted  by  the  computer.  (Skilled 
users  should  be  able  to  suppress  this  feature  if  so 
desired. ) 

Abbreviations 


6.  The  method  of  simple  truncation  should  be  used  in  forming 
abbreviations  for  terms;  e.g.,  deleting  all  but  the  first 
three  to  five  letters  of  the  words.  The  value  of  this 
technique  is  markedly  increased  when  it  is  uniformly  applied 
(with  the  possible  exception  of  words  which  have  commonly 
known  abbreviations).  Allowance  must  be  made  for  different 
words  resulting  in  the  same  abbreviation  when  truncated. 

User  understanding  of  how  the  abbreviations  are  formed  is 
valuable . 

Dialogue  Transactions 

7.  The  system's  messages  to  the  user  should  be  in  a  directly 
usable  form  and  provide  prompts  or  reminders  of  the  current 
state  of  transaction  development.  The  user  should  not  have 
to  refer  back  to  previous  transactions  in  order  to  deter¬ 
mine  the  present  states  of  the  system.  hengthy  sequences 
of  transactions  should  be  recapped  periodically. 

8.  When  the  system  displays  information,  "it  should  be  in  the 
form  needed  at  that  point  even  if  the  format  is  different 
from  that  provided  in  the  data  base  or  (from)  when  it  was 
originally  entered.  For  example,  in  a  payroll  or  cost¬ 
accounting  system  salaries  may  be  stored  in  hourly  rates, 
but  if  the  current  activity  requires  monthly  or  yearly 
rates,  the  computer  should  make  the  required  transformation 
and  display  accordingly." 
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9. 


Users  should  be  able  to  easily  modify  a  request  that.  is. 
revealed  to  be  incorrect.  In  particular,  they  should  be 
able  to  move  backwards  through  a  dialogue  sequence  in 
order  to  change  an  entry.  Introducing  such  a  change 
should  not  require  re-entry  of  all  the  correctly  entered 
material . 

10.  A  small  proportion  of  queries  usually  accounts  for  a  high 
proportion  of  the  user's  activities.  These  queries  should 
be  designed  for  greatest  ease  of  accomplishment. 

11.  Some  user  queries  require  a  long  response  time.  The  eomj at e 
should  acknowledge  the  receipt  of  a  query  and  should  later 
indicate  that  a  response  is  available. 

Specific  Recommendations: 

Formal  Query  Lanquages 

Layering 

12.  The  features  of  a  query  language  should  be  partitioned  into 
groups  or  layers.  The  easiest  layer  should  be  aide  to  stand 
alone  and  is  intended  for  users  of  limited  sophistical  ion  ■>• 
limited  need.  The  layers  should  then  increase  in  com] lexi'y 
for  use  by  more  sophisticated  personnel.  Such  a  |  rocedure 
will  broaden  the  base  of  users. 

Semantic  Confusion 


13.  Avoid  the  use  of  operators  such  as  "or  more"  and  "or  less" 
(e.g.,  do  not  require  the  user  to  convert  "over  50  years  old 
into  "51  or  more").  People  have  difficulty  using  these  oi et 
ators  correctly. 

14.  Query  language  operators  should  not  be  given  semantically 
similar  names  (e.g.,  "SUM"  and  "COUNT").  To  avoid  confusion 
operators  should  be  given  names  that  are  distinctive  and 
self-explanatory. 

Term  Specificity 

15.  For  inexperienced  users,  the  use  of  global  terms  (e.g.,  gen¬ 
eral  terms  which  subsume  a  number  of  specific  terms)  is  not 
recommended  unless  the  specific  terms  of  information  subsume 
under  the  global  terms  are  retrieved  together  frequently. 

The  availability  of  global  terms  does  increase  the  speed  of 
data  entry  (i.o.,  typing)  but  does  not  affect  accuracy. 


Specific  Recommendations: 
Natural  Query  Lanquages 


Clarification  Dialogue 


16.  Natural  query  language  systems  should  be  capable  of  carrying 
out  a  "clarification  dialogue."  Users  will  frequently  input 
poorly  stated  queries  and  it  is  not  sufficient  for  the  system 
to  simply  reject  them.  Instead,  the  system  should  be  capable 
of  guiding  the  user  through  a  dialogue  which  will  result  in 
the  formulation  of  a  proper  statement. 

Quasi-Natural  Languages 


17.  Quasi-natural  languages  should  be  considered  as  design  options 
in  situations  where  it  is  neither  possible  to  teach  a  formal 
query  language  to  potential  users  nor  is  it  feasible  to  develop 
a  natural  query  language.  Quasi-natural  languages  are  English- 
like  in  structure  but  they  are  not  capable  of  truly  "under¬ 
standing"  the  text's  meaning.  For  a  quasi-natural  language 
to  be  applicable,  the  system's  task  should  be  narrow  and  well 
defined.  Examples  of  the  use  of  a  quasi-natural  language  are 
given  in  the  text. 


6.  GLOSSARIES 

No  reports  in  this  general  category  were  reviewed. 

7.  ERROR  HANDLING 

7.1  Error  Prevention 

Mace ,  D.J.,  Harrison,  P.C.,  Jr.,  and  Seguin,  E.L.  Prevention  and  remediation 
of  human  input  errors  in  ADP  operations .  ARI  Technical  Report  No.  395,  August 
1»7  >.  (At)  At >8 1  730)  . 

This  document  goes  into  detailed  discussion  about  input  errors,  ways  to 
overcome  them,  and  procedures  for  analyzing  the  cost-benefit  of  circumventing 
them.  An  example  illustrative  of  the  thinking  process  involved  in  applying 
the  MAUM 3  cost-benefit  analysis  to  TOS  is  presented.  (Their  example  is  not 


VMAUM  is  the  Multi-Attribute  Utility  Measurement  presented  in:  Edwards,  W., 
Guttentag,  M. ,  and  Snapper,  K.  in  Handbook  of  evaluation  research,  volume  1, 
E.L.  Streuning  and  Marcia  Guttentag,  eds .  Sage  Publications,  Beverly  Hills, 
California,  1975. 
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nearly  so  concrete  as  the  one  provided  by  the  original  authors  who  apply  the 
procedure  to  the  planning  process  of  the  Office  of  Child  Development  and  use- 
real  data. ) 

Of  greater  interest,  particularly  at  this  stage  of  our  own  project  effort, 
is  the  attention  to  the  sources  of  data  input  errors  and  their  prevention  and 
detection.  Table  3  presents  the  authors'  summary  of  error  types,  causes,  and 
alternatives  for  prevention  and  detection. 

Gade,  P.A.,  Fields,  A.F.,  and  Alderman,  I.N.  Selective  feedback  as  a  training 
aid  to  on-line  tactical  data  inputting .  ARI  Technical  Paper  No.  349,  November 
1978.  (AD  A061  789). 

This  study,  one  of  a  series  focused  on  improvement  of  data  entry,  is  an 
experimental  investigation  into  the  effectiveness  of  various  instructional 
strategies  for  overcoming  error  in  data  entry.  The  intended  special  appli¬ 
cation  of  the  research  is  the  MIOD  (Message  Input  Output  Device)  operator  of 
the  Army's  Tactical  Operations  System  (TOS) . 

Earlier  research  reviewed  by  Alderman  (1975)  focused  on  improved  data 
input  (reduced  time  and  error)  by  reallocating  the  inputting  functions/tasks 
and  by  development  of  job  training  aids.  Strub  (1971)  demonstrated  increased 
MIOD  operator  accuracy  by  use  of  a  CRT  for  direct  data  input  and  decreased 
entry  time  by  use  of  a  computer-generated  format  on  the  CRT  (1975) .  He 
suggested  that  an  automated  training  program,  incorporating  input  aids  with 
a  response-sensitive  training  instructional  strategy,  would  improve  training 
effectiveness.  This  study  investigates  that  hypothesis. 

These  investigators  hypothesized  that  a  response-sensitive  instructional 
strategy,  in  conjunction  with  computer-generated  feedback,  would  have  the 
following  desirable  effects  on  MIOD  operator  training,  as  compared  to  more 
conventional  training  strategies: 

a.  More  rapid  progress  through  the  training  material. 

b.  Improved  transfer  of  training  to  operational  environments. 

Participants,  71  enlisted  personnel,  took  part  in  a  training  session 
and  a  transfer- testing  session.  Each  participant  completed  21  messages 
during  the  training  session.  In  the  testing  session,  participants  completed 


Table  3 


rror 


Types , 


Causes  and  Alternatives 


for  Prevention 


and  Detection 


ERROR  TYPE 


CAUSE  OE  ERROR 


PRFVE  NT  I  OH  AND  DEIFMII* 


OMISSION  OT  HESSACE  SET 


L*i  k  ol  knalH|r  of  u-*er /operal «*» 
Incompat  IE  I  1  |i  y  hrtvtED  AOuii  *  docu- 

r*ni  and  Input  dialogue 
Nlaplared  J.  uirnl 

Operator  failure 


tiger /operal  or  aelectlnn 

l!«n  uprntoi  training 

Kevlar  procedure*  -  horizontal  dlatrl- 
but  ton  of  Input  artlvltlea 

Ravlae  aouri  r  doument  format  a 

Revise  Input  loraatx 


OMISSION  OF  (IAEA  ELEMENT 
CROUP 


OMISSION  Of  DATA  ELEMENT 


Incoapat  lb*l  It  v  between  aource  doc u- 
mant  and  Input  dialogue 

Skip  a  Una  or  loaa  a  page 


Ravlaa  proredur**  -  vertical  dlatri 
button  of  input  artlvltlea 

Ravlaa  tourer  docuaant  format  a 

Ravlaa  Input  format  a 


Improper  presumption  of  default 

Flamanr  Inpur  Into  wrong  Inrjrlon 

Loaa  of  place  In  aour.r  or  pre¬ 
form*’ ted  dor  taient 

Loaa  of  place  in  dl*l"Ru*  with  avetem, 
a.g.  cursor  pea  It  Ion 


I'aa  fotmata  with  explicit  label* 


Dlaplav  default  value* 
Conditional  arroi  tie- king 
U>ndltt<nal  formatiing 
Interactive  dialogue 


VALID  CliOrS/ RESTRICTED  ITEMS 


Verbal  Ctrora 
(i.loaaary ) 


Incorrect  recall 
Inrorre  t  recognition 


Input  lai  guage 
Expanded  de1!rltt„na 


Transcription  error* 


Conditional,  pnbabl  1  lat  I  •  ,  or  adaptive 
error  c  he<  k  !ng 


Quantitative  Error* 


Incorrect  acale  lonveralon* 


mndlng  ut  number* 


arelrti  tranacrlpc ton. 
character  tranepoair: 
decimal  placement 


INVALID  CODES/ RESTRICTED  ITEMS 

Verbal  Errura  - 
ItKortxct  Abbreviation* 


incorrect  recall 
Incorrect  recognition 
Typographic  mistake 


Input  language 
Menu  a»l*>i  Ion 
DiapEayed  iudll 

Closest y  dlipley  ot  "HELP"  routine* 


Typographic  atatake  Format*  with  explicit  label* 

Format  generated  confusion  Date  b*»*  update 

Conditional  formatting 
Quavl ton  and  anewat  dialogue 
Menu  **l*i.i  Ion 
Displayed  rode* 


adapt  ive 


Typographic  ml  at  aka 
Former  generated  confusion 


Illegal  Entry 


Incorrect  male  conversion 
Dels  input  with  Incorrect  ecal* 
Incorrect  rounding  of  number* 
uatelea*  t  ranger  Ipt  ion .  Including 
.her  acrei  t  r  anapov  u  Ion  and 
decimal  placement 


UNRESTRICTED  HEMS 


tipai  c-i-  «|  i  .id  I  ng 

i  Sea#  i  ode  uaeil  with 


lei  or  %ifr  aeanlngai 


Coding  i  anvent tone  Inadequate 
Data  elameni  too  short 


laptop*!  tudlng  i  on v# hi  |..nl 
Lengthen  data  element 
Provide  gloeaatt  ol  ptevloua  entile* 
beta  baa*  update 
Expanded  def lull  ton* 

I  apt uve  i  odlng  < onvent Iona 
Provide  gloaaaty  ot  pievlnu*  entrlea 
bar*  baae  update 
Expanded  de I  In  1 1  Iona 
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as  many  messages  as  possible,  up  to  a  miaxmum  of  43.  Participants  were 
assigned  randomly  to  four  levels  of  training  feedback,  as  follows: 


a.  Minimum  feedback — informed  only  of  error  on  last  entry,  no 
correction. 

b.  Edit  feedback--error  message,  use  of  legal  entry  tables  from 
which  to  correct  the  error. 

c.  Remedial  feedback--error  message  displaying  both  the  incor¬ 
rect  and  correct  entry,  and  participant  error  correction. 

d.  Response-sensitive  feedback — same  as  remedial,  except  that 
after  3  correct  consecutive  entries  for  a  particular  element 
of  a  particular  type  of  format,  automatic  fill  of  the  element 
by  the  computer. 

There  were  also  two  conditions  of  transfer-testing — no  feedback  and  edit 
feedback. 

Speed  and  accuracy  were  the  performance  measures.  Response-sensitive 
training  proved  effective  in  reducing  training  time  and  was  as  effective  as 
other  training  strategies  in  reducing  error  rates.  However,  since  the  study 
does  not  suggest  that  response-sensitive  training  will  assist  in  catching 
errors  that  escape  error  detection  routines  and  does  not  improve  the  error 
rate,  the  decision  of  whether  or  not  to  implement  response-sensitive 
training  should  be  based  on  the  trade-off  between  the  cost  of  training 
development  and  the  benefit  of  reduced  training  time. 

7.2  Error  Detection 

No  reports  in  this  category  were  reviewed. 


7.3  Error  Feedback 


No  reports  in  this  category  were  reviewed. 

7.4  Error  Correction/Recovery 

No  reports  in  this  category  were  reviewed. 
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8.  USER/OPERATOR  CONFIGURATION 


Fink,  C.D.,  and  Carswell,  W.A.  Integrated  personnel  and  training  information 
for  TRADOC  system  managers  (TSM).  Technological  Caps.  ARI  Research  Report 
No.  1238,  February  1980. 

(The  authors'  abstract  follows.) 

The  Integrated  Personnel  Support  (IPS)  model  outlines  the  procedures 
that  should  be  followed  during  the  development  of  personnel  and  training  sub¬ 
systems  for  new  materiel  systems.  This  report  identifies  some  of  the  tech¬ 
nological  problems  that  must  be  solved  before  benefits  from  the  application 
of  the  IPS  model  can  be  achieved.  These  problems  or  "technological  gaps" 
were  identified  during  an  extensive  literature  review  for,  and  the  subsequent 
development  of,  a  handbook  for  TRADOC  System  Managers  (TSM) . 

Most  of  the  technological  gaps  related  to  deficiencies  in  procedures  for 
estimating  training  and  personnel  requirements,  and  for  the  development  of 
training  strategies,  during  Phases  I  and  II  of  the  Army's  Life  Cycle  System 
Management  Model  (LCSMM) .  Specifically,  there  was  found  to  be  an  absence 
of  satisfactory  techniques  for  (a)  deducing  training  and  personnel  require¬ 
ments  from  materiel  characteristics;  (b)  identifying  excessive  human  resources 
demands  stemming  from  materiel  concepts;  (c)  identifying  functions  most  appro¬ 
priately  performed  by  equipment,  by  persons,  or  by  a  man-machine  combination; 
(d)  generating  task  analytic  data  during  Phase  I  of  the  LCSMM;  (e)  identify¬ 
ing  high-risk  training  tasks  during  Phase  I  of  the  LCSMM;  (f)  indicating 
appropriate  training  strategies  before  actual  hardware  is  developed;  and 
(g)  describing  human  resources  data/requirements/constraints  in  terms  that 
are  meaningful  to  design  engineers. 


Additional  ARI  Literature  to  be  Reviewed 


Alderman,  I.N.  Tactical  Data  Inputting:  Research  in  Operator  Performance 
and  Training.  Proceedings  of  the  Second  National  Symposium  on  the  Manage¬ 
ment  of  Data  Elements  in  Information  Processing.  Washington,  D.C.:  NBS/ 
ANSI,  1976. 


27 


i. '  i  c  c  i  one ,  D.C.  ,  Garnet  ,  M .  •  I .  ,  aiui  Channon,  J  .  B.  A  Framework  for  file  Develop¬ 
ment  of  Improved  Tactical  Symbology.  Aii  1  Technical  lii'inrt  No.  -40  3,  August. 

1  (AD  A07e  el  7) 

Colson,  K.K.,  Freeman,  F.S.,  Mathews,  I.. I’.,  and  Stott  lor,  ,7.  A.  Development 
of  an  Informational  Taxonomy  of  Visual  Displays  for  Army  Tactical  Data  Sys¬ 
tems.  ASi!  hit wt.i.’v/i  Mt'mornndum  .Vo.  74-4.,  February  1974.  (AD  Au82  95  1} 

aanda,  T.M.  A  Comparison  between  a  Standard  Map  and  a  Reduced  Detail  Maj 
wi  til  in  a  Simulated  Tactical  opera  t  ions  System  (SIMTtS).  Alii  Tt  \:hn  i  c.t 
Pie:  .Vo.  274,  June  l"7e.  (AD  A028  752) 

Sranda,  T.M.  An  Evaluation  of  Visual  Search  Behavior  on  a  Cathode  Kay  Tube 
ft  i  1  izina  the  Window  Technique.  ARI  Technical  Paper  .Vo.  293,  February  l‘J7ri. 
(AD  AOS 3  352) 

ileminqway,  F.W.,  and  Kubala,  A.L.  A  Comparison  of  Speed  and  Accuracy  of 
Interpretation  of  Two  Tactical  Symbologies.  ART  Technical  Report  .Vo.  189, 
July  1979.  (AD  A07  5  428) 

ileminqway,  F.W.,  bubal.  \ .  A.L.  and  Chastain,  G.D.  Study  of  Symbology  for 
Automated  Graphic  Displays.  ARI  Technical  Report  Xo.  TR-79-A2R ,  May  1979. 

(AD  At) 7 b  916) 

Mawrocki,  L.R.  Word  Abbreviations  in  Man-Computer  Communication  Systems. 

Mil  Working  Paper  III'  79-04 .  June  1970. 

Modiste,  B.R.,  Michel,  R.R. ,  and  Stevens,  J.W.  Initial  Strategies  for  the 
Tactical  Operations  System  (TSO)  Support  of  the  Command  and  Control  Process. 
ARi  Technical  Report  TR-7 S-A16 ,  June  1978. 

Moses,  F.L.,  and  Maisano,  R.E.  User  Performance  Under  Several  Automated 
Approaches  to  Changing  Displayed  Maps.  ARI  Technical  P.reer  .Vo.  Idu ,  June 
1979.  (AD  AO 7 3  726) 

Neqroponte,  N. ,  llerot,  C. ,  and  Weinzapfel ,  G.  One  point  touch  input  of 
vector  information  for  computer  displays.  ART  Technical  Report  .Vo.  TR-7R-T! i' 
( NT I S  No.  ADA064278) 

Patten,  S.M.  An  Inductive  Taxonomy  of  Combat  Intelligence  Data.  ART  Researc 
Memorandum  74-14-  December  1974.  (AD  A076  802) 

Potash,  I..M.  Effects  of  Retrieval  Term  Specificity  on  Information  Retrieval 
From  Computer  Based  Intelligence  Systems.  ,\R1  Technical  Paper  ,Vo.  37'), 

July  1979.  (AD  A072  312) 

Sidorsky,  R.C.  Alpha-Dot:  A  New  Approach  to  Direct  Computer  Entry  of  Battle 
field  Data.  ART  Technical  Paper  No.  249,  January  1974.  (AD  774  841) 

Sidorsky,  R.C.  Source  Data  Automation  via  the  Alpha-Dot  Tablet:  A  Feasi¬ 
bility  Study.  ART  Working  Paper  No.  79-07.  June  1979. 

Sidorsky,  R. ,  Gellman,  L.H.,  and  Moses,  F.P.  Survey  of  Current  Developments 
in  Tactical  Symbology:  Status  and  Critical  Issues.  ARI  Workina  Paper  .Vo. 
HP-79-03,  May  1979. 
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Siegel,  A. I.,  Leahy,  W.u.,  and  Wolf,  J.J.  A  Computer  Model  for  Simulation 
of  Message  1’rocessina  in  Military  Hxercise  Control  and  Kvaluat ion  Syst  cms. 

Ak!  Techn  i  c.i  I  Kt'port  TR-77-A22  .  October  1977. 

St  rub,  M.  II.  Evaluation  of  Man-Computer  Input  Technique;;  for  Military  Infor¬ 
mation  Systems.  AR1  7  \'chnical  Resea  rch  Note  .Vo.  JJo ,  May  1071. 

St  rub,  M.H.  Automated  Aids  to  on- Id  no  Tact  ical  Lata  Inputt.  inq.  Ak !  V<  i  - 
cai  /’./per  .Vo.  Jiv’,  Pebruary  1071.  (Ab  Aula  P>o) 

NON-ARI  LITERATURE 

Literature  in  the  public  domain  was  explored  for  the  purpose  of  finding 
information  appropriate  to  the  development  of  guidelines  and  criteria  and, 
indeed,  to  identify  useful  sources  of  guidelines  and  criteria.  This  section 
of  this  report  is  not  intended  to  be  a  "review"  of  the  literature  in  the  tra¬ 
ditional  sense.  Rather,  its  purpose  is  to  identify  documents  which  provide 
human  factors  guidelines  and  evaluation  criteria  for  the  design  of  user/ 
operator  transaction  in  battlefield  automated  systems  or  which  could  substan¬ 
tively  contribute  to  the  development  of  such  guidelines  and  criteria. 

Cagne,  R.M.,  et  a 2 .  Psychological  principles  on  system  development,  Holt, 
Rinehart  and  Winston,  Now  York,  1963. 

Tii is  volume  remains  a  basic  and  often  quoted  source  of  information  for 
system  design.  Its  emphasis  on  the  human  component  of  the  system,  and  parti¬ 
cularly  on  the  development  of  that  component,  is  especially  cogent  to  the 
objectives  of  this  project.  In  addition  to  providing  much  information  concern 
ing  the  design  of  the  human-machine  interface,  the  text  will  be  particularly 
useful  to  the  project  in  defining  the  appropriateness  of  guidelines  for 
different  stages  of  system  development. 

Ramsey,  H.R.  and  Atwood,  M.E.  Human  factors  in  computer  systems:  A  review  e; 
the  literature.  Science  Applications,  Inc.,  Englewood,  Colorado,  21  Sopembor 
1979.  (Technical  Report  SAI-79-111-DEN) . 

Ramsey,  H.R.,  Atwood,  fl.E. ,  and  Kirshbaum,  P.J.  A  critically  annotated  biblio 
graphy  of  the  literature  of  human  factors  in  computer  systems,  Science  Applica 
tions,  Inc.,  Englewood,  Colorado,  31  May  1978.  (Technical  Report  SAT-78-07i>- 
DEN)  . 


Ramsey  and  Atwood  conducted  a  broad  survey  of  the  literature  on  human 
factors  in  computer  systems  to  assess  the  state  of  knowledge  in  the  area. 
Their  objectives  were  to: 

a.  Determine  whether  or  not  that  knowledge  is  adequate  to  support 
the  development  of  a  human  factors  guide  to  the  design  of  inter¬ 
active  computer  systems. 

b.  Present  a  descriptive  and  critical  literature  review  with 
respect  to  the  issue  of  design  guidelines. 

Their  conclusions  with  respect  to  the  feasibility,  potential  utility,  a 
possible  form  and  content  of  a  human  factors  guide  to  interactive  computer  s 
tern  design  are  as  follows: 

a.  The  existing  literature  relevant  to  this  field  is  badly  frag¬ 
mented  because  of  its  foundation  in  several  different  disci¬ 
plines,  and  because  relevant  empirical  data  include  those 
derived  from  many  studies  not  specifically  dealing  with 
computer  systems.  Much  of  this  literature  is  outside  of 
that  normally  considered  by  human  factors  personnel,  and 

the  vast  majority  is  outside  the  range  used  by  interactive 
system  designers.  There  is  a  strong  need  for  the  develop¬ 
ment  of  integrated  guidelines. 

b.  While  there  is  a  large  body  of  empirical  data  relevant  to 
such  guidelines,  there  are  many  significant  gaps.  Ir.  par¬ 
ticular,  there  is  inadequate  information  to  support  the 
development  of  highly  quantitative  "reference  handbook"- 
type  guidelines,  except  within  certain  fairly  limited  sub- 
domains  . 

c.  Consideration  of  the  problem-solving  behavior  and  information 
needs  of  the  interactive  system  designer  leads  us  to  believe 
that  "reference  handbook"  guidelines  would  not  truly  satisfy 
the  need  anyway.  What  is  needed  is  a  design  guide  which  is 
largely  procedural  in  nature  and  is  organized  around  the 
design  process  employed  by  designers. 

d.  Despite  the  existing  gaps  in  our  knowledge,  a  design  guide 
of  this  sort  appears  feasible.  Such  an  approach  is  compat¬ 
ible  with  the  presentation  of  human  factors  methods,  as  well 
as  empirical  data  and  specific  recommendations.  In  such  a 
presentation,  general  psychological  knowledge  can  often  be 
used  to  advantage,  especially  in  areas  in  which  empirical 
information  is  sparse.  In  areas  in  which  specific  recom¬ 
mendations  are  impossible,  this  approach  can  at  least 
direct  the  designer's  attention  to  relevant  factors. 
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>lu  idol  irift;  associated  wit.h  the  early  system  design  j  loeern 
(e.g.,  user  requirements  analysis)  will  necessarily  empha¬ 
sise  methods  to  Lo  employed  by  the  designer.  hater,  whoti 
tiro  design  decisions  are  more  concrete  and  detailed — arid 
concern  at  eas  in  wiiich  more  empirical  research  has  been 
done--the  guidelines  can  be  more  specific  and  prescriptive. 

f.  Alt'nouq’n  it  is  feasible  to  construct  a  design  guide  for 

interactive  systems  in  general,  it  may  be  better  to  develop 
them  for  restricted  types  of  systems  (e.g.,  message  pro¬ 
cessing,  or  tactical  information  systems).  User  behavior, 
and  thus,  desirable  system  properties,  tend  to  be  highly 
task-specific.  By  concentrating  on  a  restricted  range  of 
user  tasks,  it  should  be  possible  to  make  guidelines  more 
prescriptive  and  explicit,  and  to  use  more  meaningful 
examples,  as  well  (pp.  141-143). 

The  annotated  bibliography  was  searched  for  sources  of  guidelines  appro¬ 
priate  to  the  design  of  usor/operator  transactions  in  battlefield  automated 
systems  to  be  more  fully  developed  in  the  next  phase  of  the  project.  In 
Table  4,  these  sources  are  identified  within  the  classification  scheme  of 
design  features  presented  in  Table  1. 


Table  4 

Sources  of  Guidelines  Relevant  to  User/Operator  Transactions 


1.  CONTROL  METHODS 


Author ( s) 

Bennett,  J.L.,  1972 
Chamberlain,  R.G.,  1975 
Cheriton,  D.R.,  1976 
Dinter ,  H. ,  1969 

Doner ty,  W.J.,  Thompson,  C.M., 
&  Boies,  S.J.,  1972 

Kennedy,  T.C.S.,  1974 
Martin,  T.H.,  &  Parker ,  E.B., 
1971 

Sterling,  T.D.,  1974 
Sterling,  T.D.,  1975 
Stewart,  T.F.M.,  1976 
Vaughan,  W.S.,  dr.  t.- 
Mavor,  A.S. ,  1972 
Wood,  R.C.,  1972 


Specific  Topic 

Interactive  systems 
Interactive  system  design 
Man-computer  dialogue 
Computer-based  information 
systems 

Interactive  systems--functions , 
software,  linguistics, 
scheduling 

Interactive  dialogue 
Interactive  systems 

Computers  in  general 
Computers  in  general 
Man-computer  interface 
Functional  properties  of  inter¬ 
active  systems 
Interactive  systems 


1.1 


Command  La nguages 


No  sources  of  guidelines  specific  to  command  languages  were  cited. 


1  . 2  Menus 

No  sources  of  guidelines  specific  to  menus  were  cited. 


1  .  3  Function  Keys 

Dolotta,  T.A.,  1970 
Hanes,  L.F. ,  1975 
Yllo,  A. ,  1962 


1  .4  Hybrid  Methods 

No  sources  of  guidelines  specific  to  hybrid  methods  were  cited. 

1 . 5  Prompts/HELPS 

No  sources  of  guidelines  specific  to  prompts/HELPS  were  cited. 


2.  DISPLAY  FORMAT 


Danchak,  M.M.,  1976 
Gould,  J.D. ,  1968 
Grether,  W.F.,  &  Baker,  C.A 
1972 

Ton,  W.H.,  1969 
Whitham,  G.E.,  1965 


2 . 1  Fixed  Alphanumeric  Displays 

Buckler,  A.T.,  1977  Alphanumeric  displays 

Hayman ,  E. ,  1969  CRT  alphanumeric  displays 

2 . 2  Variable  Length  A1 phanumeric  Displays 

No  sources  of  guidelines  specific  to  variable  length  alphanumer i c  di 
plays  were  cited. 


Display  design 
CRT  display — visual  factors 
Visual  information  {presenta¬ 
tion 

Displays 

Display  size/resolution 


Teletypwriter  terminal 
Keyboards 

Keypunch-data  entry 
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2 . 3  Graph  ijy  Di_s_pJ  ays 

Barmack,  9.K.,  i»  Sinajko,  H.W.,  Interactive  graphics 

1-K'O 

Foley,  J.E.,  k  Wallace,  V.L. ,  Interactive  graphics 

1974 


2.4  Hi ghl i ghting 

Christ,  R.E.,  1975 
Christ,  R.E.  &  Corso,  G.M. , 
1975 


Color  coding-visual  displays 
Color  coding-visual  displays 


3.  DATA  ENTRY  AND  HANDLING 


Wallace,  V.L.,  1976  Graphical  input  devices 

4.  MESSAGE  COMPOSITION  AIDS 

Green,  E.E.,  1976  Message  design 

4 . 1  System  Design  Features 

Bryden,  J.E.,  1969 
Cropper,  A.G.,  &  Evans,  S.J.W., 

1968 

Jenny ,  J . A. ,  197 3 
Stewart,  T.F.M.,  Oestberg,  O.  , 

&  MacKay,  C.J.,  1974 

4 . 2  Format  for  Alphanumeric  Messages 

No  sources  of  guidelines  specific  to  format  for  alphanumeric  messages 
were  cited. 

4.3  Graphic  Messages 

No  sources  of  guidelines  specific  to  graphic  messages  were  cited. 


CRT  displays 
Display  design 
Display  design 
Terminals 
Terminals 


J 
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5. 


DATA  RETRIEVAL  ASSISTANCE 


No  sources  of  guidelines  specific  to  data  retrieval  assistance  were 
cited. 

6.  GLOSSARIES 

No  sources  of  guidelines  specific  to  glossaries  were  cited. 

7.  ERROR  HANDLING 

7.1  Error  Prevention 

No  sources  of  guidelines  specific  to  error  handling  were  cited. 

7 . ?  Error  Detection 

No  sources  of  guidelines  specific  to  error  detection  were  cited. 

7.3  Error  Feedback 

No  sources  of  guidelines  specific  to  error  feedback  were  cited. 

7.4  Error  Correction/Recovery 

Varley,  T.C. ,  1969  Error  correction  checklist 

8.  USER/OPERATOR  CONFIGURATION 

Smith,  S.L.,  1974  User  interface 


Engle,  S.E.,  and  Granda,  R.E.  Guidelines  tor  man/display  interface's,  IBM 
Poughkeepsie  Laboratory,  Poughkeepsie,  New  York.  19  December  1975.  (TR 
00.2720) . 


This  document  is,  to  quote  Shneiderman,  "the  best  detailed  guide  for 
design  of  interactive  systems  (op  cit.,  page  254).  The  guidelines  are 
specifically  to  assist  a  software  developer  in  designing  the  display  inter¬ 
face  between  a  program  and  intended  users  but  the  authors  also  feel  that  the 
guidelines  will  be  useful  to  hardware  developers  as  well.  There  is  much  infor¬ 
mation  which  has  been  of  use  to  this  project  in  initiating  the  development 
of  a  set  of  guidelines,  especially  since  there  is  a  focus  on  the  user  require¬ 
ments.  The  authors  recoonize  the  limitations  of  these  guidelines: 

The  present  set  of  guidelines  are  not  exhaustive.  They 
represent  a  first-level  effort  to  collect  and  identify 
what  is  known  or  could  reasonably  be  expected  to  be 
deduced  from  the  present  state  of  knowledge.  They  also 
provide  some  insight  into  those  areas  which  are  poorly 
known  or  for  which  little  factual  data  exists,  (page  1) 

Fortunately,  the  organizational  structure  adopted  by  Engel  and  Granda, 
while  not  consistent  with  or  as  comprehensive  as  that  developed  for  this 
project,  is  compatible  with  the  organization  shown  in  Table  1.  Thus,  much 
of  what  these  authors  have  presented  is  directly  applicable  to  this  project's 
purposes. 

Smith,  S.L.  Requirements  definition  and  design  guidelines  for  the  man-machine 
interface  in  C  system  acquisition .  Mitre  Corporation,  Bedford,  Massachusetts, 
31  December  1979.  (Mitre  Technical  Report  MTR-3888) . 

This  report  explores  the  status  and  needs  of  the  man-machine  interface 
(MMI)  and  comes  to  the  following  conclusions  and  recommendations: 

This  report  proposes  that  a  significant  program  of  work  be 
undertaken.  It  marks  a  beginning,  not  an  end.  Thus  only 
tentative,  interim  conclusions  are  offered,  and  the  chief 
recommendation  is  that  the  work  be  carried  forward.  To  follow 
the  approach  outlined  here,  the  next  steps  involve  further 
development  of  tools  for  MMI  requirements  definition  and 
design,  and  evaluation  of  the  application  of  those  tools 
in  C3  system  acquisition. 
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Three  kinds  ot  tools  have  been  proposed: 


1.  An  Mill  requirements  matrix,  a  systematic  tabulation 
of  the  functional  capabilities  required  by  different  oper¬ 
ator  tasks,  to  permit  requirements  definition  in  advance 
of  MMI  desiqn.  The  concept  of  a  requirements  matrix  is 
described  in  Section  2  of  this  report  and  illustrated  in 
Appendix  B.  A  tentative  conclusion  is  that  this  require¬ 
ments  matrix  could  prove  a  useful  tool,  at  the  least  pro¬ 
viding  a  check  list  for  systematic  consideration  of  MMI 
requirements,  with  some  potential  promise  as  an  effective 
means  of  coordinating  analysis  with  design.  Further  develop¬ 
ment  of  this  requirements  matrix  is  recommended  in  conjunction 
with  evaluation  in  system  acquisition. 

2.  MMI  design  guidelines,  a  compilation  of  available  wisdom, 
to  be  tailored  to  the  requirements  of  different  system  appli¬ 
cations.  Design  guidelines  are  discussed  in  Section  3  of  this 
report  and  illustrated  in  Appendix  C.  A  tentative  conclusion 
is  that  guidelines  can  be  found  for  the  design  of  certain 
common  MMI  functional  capabilities,  and  that  these  can  be 
related  at  least  approximately  to  the  requirements  matrix, 
with  potential  for  tailoring.  It  is  recommended  that  the 
sample  set  of  guidelines  presented  here  be  exp>anded  to  pro¬ 
vide  broader  functional  coverage,  and  then  evaluated  in 
system  acquisition. 

3.  Design  documentation,  some  means  of  specifying  detailed 
MMI  design  for  coordination  and  review  in  advance  of  soft¬ 
ware  implementation.  Design  documentation  is  discussed  in 
Section  4  of  this  report,  and  one  possible  approach  is 
illustrated  in  Appendix  D.  A  tentative  conclusion  is  that 
such  documentation  could  prove  useful  for  both  initial  design 
and  subsequent  design  modification,  but  that  it  is  not  clear 
just  what  are  the  proper  means  for  imposition  of  this  special 
documentation  requirement  in  system  acquisition.  This  ques¬ 
tion  should  be  explored  further,  until  a  formal  documentation 
requirement  can  be  developed. 

All  of  these  proposed  tools  must  be  applied  to  assess  their 
value.  Discussion  of  the  possible  utility  of  these  tools  for 
MMI  design  is  an  interesting  exercise  in  armchair  philosophy, 
but  will  have  no  practical  effect  unless  carried  forward  and 
tested  in  actual  system  development.  Certainly  no  Air  Force 
or  DoD  design  standard  can  be  justified  on  the  basis  of  this 
discussion  alone,  without  practical  evaluation  (pp.  40  and  41). 

One  of  the  Appendices  to  this  rejx>ri  contains  a  set  of  design  guidel 
Table  5  presents  a  sample  of  these  guidelines  which  will  be  considered  in 
this  project's  further  development  of  guidelines  and  criteria. 


Table  D 


Design  Guidelines  for  Data  Entry  Functions 


Code 

2.0  DATA  ENTRY/INPUT 

-  1  Where  data  entry  is  a  significant  task  function,  it  should  bo 

accomplished  via  the  operator's  primary  display.  (For  example, 
entry  via  typewriter  is  acceptable  only  if  the  typewriter 
itself,  under  computer  control,  is  the  primary  display  medium.) 

-  2  Data  entry  transactions,  and  associated  displays,  should  be 

designed  so  that  the  operator  can  use  one  mode  of  entry  as 
long  as  possible  before  having  to  switch  to  another  (e.q., 
switching  from  lightpen  to  keyboard  input) . 

-  3  Except  for  passwords  and  other  secure  entries,  keyed  inputs 

should  always  appear  in  the  display. 

-  4  Keyed  data  entry  and  change  on  an  electronic  display  should 

generally  be  accomplished  by  direct  character  replacement, 
in  which  keyed  inputs  replace  underscores  or  previous  entiles 
(including  default  values)  in  defined  data  fields. 

-  5  Wherever  possible,  data  entry  should  be  self-paced,  depending 

upon  the  operator's  needs,  attention  span  and  time  available, 
rather  than  computer  processing  or  external  events.  (Where 
self-pacing  does  not  seem  feasible,  the  general  approach  to 
task  allocation  and  MMX  design  should  be  reconsidered.) 

-  6  Using  a  form-filling  dialogue,  entry  of  logically  grouped 

items  should  be  accomplished  by  a  single,  explicit  action 
at  the  end,  rather  than  requiring  separate  entry  of  each 
i  t  cm . 


Smith,  S.L.  Man-mac  hi nc  interface  (MMI)  requirements  definition  and  desiqn 
quidtd ines :  A  progress  report.  Mitre  Corporation,  Bedford,  Massachusetts, 
30  September  1980.  (Mitre  Technical  Report  MTR-8134). 


This  document  extends  the  work  previously  presented  by  Smith.  Of  partic¬ 
ular  interest  to  the  project's  further  development  of  design  guidelines  and 
criteria  are  an  Appendix  which  presents  design  guidelines  for  sequence  control 
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and  another  Appendix  which  presents  a  MMI  requirements  checklist.  The  latter 
is  assumed  to  have  potential  for  the  designation  of  design  principles  appro¬ 
priate  to  different  stages  of  system  design.  Examples  of  each  set  of  data 
are  presented  below  in  Table  6  and  7. 

Shneiderman,  Ben.  Software  psychology :  Human  factors  in  computer  and  infor¬ 
mation  systems,  Winthrop  Publishers,  Inc.,  Cambridge,  Massachusetts,  1980. 

This  text  has  become  a  "standard  for  the  industry"  in  a  very  short  time. 

It  is  addressed  to  a  broad  and  varied  audience:  professional  system  designers, 
managers,  and  programmers;  advanced  undergraduate  and  beginning  graduate  stu¬ 
dents  in  human  factors  in  computers  and  information  systems;  and  industrial 
and  academic  researchers  in  computer  science,  information  systems,  psychology, 
and  human  factors.  The  book  provides  a  review  of  a  large  number  of  current 
research  topics  relevant  to  the  development  of  design  guidelines  and  criteria, 
for  user/operator  transactions  with  battlefield  automated  systems.  The 
author's  intent  to  provide  a  psychological  perspective  to  permit  increased 
awareness  of  the  distinctions  between  people  and  computers  has  particular 
advantage  for  this  project's  purposes. 

Shneiderman  defines  software  psychology  as  "the  study  of  human  perfor¬ 
mance  in  using  computer  and  information  systems"  (page  5)  and  identifies  the 
foci  of  software  psychology  as  "ease  of  use,  simplicity  in  learning,  improved 
reliability,  reduced  error  frequency,  and  enhanced  user  satisfaction"  (ibid). 

Of  most  relevance  to  this  project  are:  (a)  the  replication  of  and  development 
of  some  specific  guidelines  for  programming  and  systems  design,  and  (b)  a 
"practitioner ' s  summary"  which  is  incorporated  into  each  chapter  of  the  book. 

Additional  Non-ARI  Literature  to  be  Reviewed 

In  addition  to  documents  previously  cited  (pages  27  -  29)  as  relevant  to 
this  project,  the  documents  cited  on  page  41  are  judged  to  have  potential 
utility  and  will  be  reviewed  as  part  of  a  formal,  extensive,  and  substantive 
literature  review. 


38 


Table  6 


Design  Guidelines  for  Sequence  Control 


SEQUENCE  CONTROL 


Objectives : 

Minimized  control  actions  by  user. 

Low  memory  load  on  user. 

Consistency  of  control  actions. 

Compatibility  of  sequence  control  with  user  needs. 
Flexibility  of  sequence  control. 


3.0  GENERAL 

-  1  Flexible  means  of  sequence  control  should  be  provided  so  that 

the  user  can  accomplish  necessary  transactions  involving  data 
entry,  processing,  retrieval  and  transmission,  or  can  obtain 
guidance  as  needed  in  connection  with  any  transaction. 

Example:  In  scanning  a  multi-page  display  the  user  should 

be  able  to  go  forward  or  back  at  will;  if  the  MM I  design 
permits  only  forward  steps,  so  that  the  user  must  cycle 
through  the  entire  display  series  to  reach  a  previous  page, 
that  design  is  deficient. 

Comment:  Necessary  transactions  should  be  defined  in  task 

analysis  prior  to  software  design. 

-  2  Control  inputs  should  be  simplified  to  the  maximum  extent 

possible,  particularly  for  tasks  involving  real-time  response, 
and  should  permit  completion  of  a  transaction  sequence  with  the 
minimum  number  of  control  actions  consistent  with  user  abilities. 

Example:  The  user  should  be  able  to  print  a  display  di¬ 

rectly  without  having  to  take  a  series  of  other  actions 
first,  such  as  calling  for  the  display  to  be  filed,  spec¬ 
ifying  a  file  name,  then  calling  for  a  print  of  that  named 
file. 

Comment:  The  software  designer  should  program  the  computer 

to  handle  intervening  steps  automatically,  informing  the 
user  what  has  been  done  if  that  seems  necessary. 


39 


Table  7 


MM  I  Requirements  Checklist 

Task  _  Reviewer  _ 

Requirement 

estimate* 

MM  I  Capability  E  U  N  Comment 

1.0  DATA  ENTRY/INPUT 

1.1.  Position  Designation 

1.  arbitrary  positions  .  .  .  ______ 

1  discrete  _  _  _ 

2  continuous  _  _  _ 

2.  predefined  positions  .  .  -  ______ 

1 .  HOME 

1  upper  left  _  _  _ 

2  center  _  _  _ 

3  lower  right  _  _  _ 

4  other  _ 

2  command  entry  area 

3  end  of  file 

4  other 

3.  incremental  positions  .  .  .  ______ 

1.  by  character 

1  right  _  _  _ 

2  left  _  _ 

3  up  _  _  — 

4  down  _  _  _ 

2.  by  interval  (TAB) 

1  horizontal  _  _  __ 

2  vertical  _  _  _ 

3.  by  other  features 

1  word  __  _  _ 

2  line  _  _  _ 

3  paragraph  _  _  _ 

4  other  _  _  _ 

1.2.  Direction  Designation 

1  vector  rotation  _  _  _ 

2  sequential  pointing  _  _  _ 

3  numeric  entry  _  _  _  _____ 

4  other  _  _  _  _____ 


Date 


*E  =  Essential,  U  =  Useful,  N  =  Not  Needed 


40 


Additional  Non-Akl  1 .  i  t  rut  urc  to  bo  koviowo.i 


Borko,  H.  Util  illation  o;  en-i  ine  inti.'tJL't  i.n.s  (Report  No.  ;  i  ■ — . : r-  /  ' 

Santa  Monica,  California:  System  Development  Cot  ;  .  ,  Auaust  1  »  • . . 

(AD  640652) 

Carbonell,  J.R.  ,  Klkin.U  J.I.,  and  Nickerson,  K,£.  n  t  :,<  i  sy.  hul  or;  \  ca  1 

importance  of  time  in  a  t  ime-shar  i  ng  .system.  Hunm  t  dor  .o . : ,  1  •  ••  ,  1",  I  U.- 

Carlisle,  J.H.  Han-compi/t  or  interactive  problem  so  Ivin.::  Pi- 1  at  ionsi, is 
bi'twecn  user  char act « >r  ist  ics  a  tit!  interlace  complex  :  t  •;  C  •  «•*  oral  'is.  -.ertat  i<> 
Yale  University,  1674).  (University  Microfilms  No.  74-. ’57. '5;  };  imi  l.u  :  at  • 
(AD  786466. ) 

Chapanis,  A.  Interactive  human  communication.  Scient  i  f  ic  Amo:  icat.,  Moc:. 
1675,  2.12(3),  36-42. 

Chapanis,  A.  Interactive  human  commas  i  cat  ion:  Some  1  e:  learned  t  tvr 

laboratory  experiments.  ’Wvt  presented  at  NAT'  Advanced  U!  idy  in.  t  it  .:, 
on  Man-Computer  Interaction,  Mali,  ;  recce ,  Se{  t ember  b’7*  . 

Foley,  J.D.  and  Wallace,  V.h.  The  art  of  natural  at. chic  m.in.-ma.  -h  i  :u- 
versation.  Proceed  huts  of  the  IFFF,  1*»74 ,  (.2,  462-471  . 

Martin,  J.  Design  of  mon-con;  u  t  <  c  .oeu<  -s .  1  non  no  .od  Clitts,  New  Jersey 

Prentice-Hall,  1673. 

Miller,  L.A.  and  Thomas,  J.u.  Jr.  Behavioral  issues  in  the  use  of  inter¬ 
active  systems  (Technical  Report  No.  RC-b32b) .  York  town  F.'iqhts,  New  York: 
IBM  Watson  Research  Center,  December  1670.  OR:  Internationa}  .Sou  null  of 
Han-Machine  Studies,  1677,  ,  506-536. 

Miller,  R.B.  Response  time  in  man-computer  conversat ional  transact  ions . 
AFiPS  Conference  Proceedings,  1668,  33  (pt.  1),  207-277. 

Nickerson,  R.S.  and  Pew,  'K.t-r.  Oblique  steps  toward  the  human  factors 
engineering  of  interactive  computer  st/s  terns  (Report  No.  21'»0)  .  Cambridge, 
Mass:  Bolt,  Beranek  and  Newman,  Inc.,  July  1671.  (Also  published  as  an 

appendix  in  M.C.  Grignetti,  D.C.  Miller,  R.S.  Nickerson,  and  K.W.  Pew, 
Information  processing  models  and  computer  aids  for  human  per formatter . 
Cambridge,  Mass:  Bolt.,  Beranch,  and  Newman,  Inc.,  June  1671). 

(AD  932913) 

Pew,  R.W.,  Rollins,  A.M.  ,  and  Williams,  C. . A .  Generic  man-computer  di.tloque 
specification:  An  alternative  to  dialogue  specialists.  In  Piweed :  nas , 

bth  Congress  of  the  [ntornation.il  Ergonomics  Association-  Santa  konira, 
California:  Human  Factors  Society,  1670,  251-254. 

Strubs,  M.H.  Automated  aids  to  on-line  tactical  .lata  inputt ing  (Technical 
Paper  202).  Arlington,.  Virginia:  U.S.  Army  Research  Institute'  for  the 
Behavioral  and  Social  Sciences,  February  1675.  (AD  A010350) 

Wallace,  V.L.  The  semen ties  of  graphic  input  devices.  In  Proceedings,  ACM 
Symposium  on  Graphic  banouages,  7’M.V  dot  ices,  Juno  b>7o,  11(e),  i  !-er>. 
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